Running easyb stories in a Gradle build

Recently, I've felt like I've been fighting Maven for all but the simplest builds, so I've been exploring alternative build systems, notably Gradle. One thing, too, that's become really important to my build processes lately is running easyb tests, so I wanted to figure out how to use easyb in my Gradle builds.

One of the nice things about Gradle is it makes good use of existing Ant tasks (why reinvent the wheel?), so integrating easyb is not difficult at all. The first step is adding the easyb artifact to the project dependencies. Unless your project is actually making use of easyb internally, you'll likely want to include it as a "testRuntime" dependency. The easyb artifact actually requires commons-cli, too, so you'll have to do one of two things:

• If you have your Gradle build set up to resolve transitive dependences with a Maven pom or Ivy file, you can set up your dependency definition to resolve transitive dependencies.


• If you're including the easyb jar directly via some other means, you can add a second "testRuntime" dependency for commons-cli.

The dependencies section might end up looking something like this:

// Access via Maven repo, 0.9.6 is currently the latest available
testRuntime("org.easyb:easyb:0.9.6@jar") { transitive = true }

// Access via local artifacts
testRuntime "org.easyb:easyb:0.9.7@jar"
testRuntime "commons-cli:commons-cli:1.2@jar"

With the dependencies in place, it's next a matter of configuring easyb to run in a Gradle task. Again, there are two options here:

• You can set up a separate task to run your easyb stories if you want to keep them separate.


• You can add the easyb run to an existing task like "check" if you know you always want to run your easyb stories whenever other tests are run.

In either case, you first need to add the easyb task via Ant's "taskdef":

ant.taskdef(name: "easyb", classname:"org.easyb.ant.BehaviorRunnerTask", 
                classpath: sourceSets.test.runtimeClasspath.asPath)

Next, you set up the easyb Ant task, telling it the classpath to use, where to put the reports and how to format them, along with where to find the behaviors you want to run:

ant.easyb( classpath: sourceSets.test.runtimeClasspath.asPath ) {
        // assume testResultsDir has already been created
        report( location:"${project.testResultsDir}/story.txt", format:"txtstory" )
        behaviors( dir: "src/test/stories" ) {
            include( name:"**/*.story" )
        }
   }

The "format" attribute can have a value of "html", "xml", "txtstory", or "txtspecification". For more information about those output formats, see the easyb documentation. Note, at this time, it doesn't appear possible to output multiple formats at once; if you require multiple formats, one workaround, depending on how extensive your tests are, might be to run easyb multiple times, once for each output format you want.

Finally, the easyb Ant task provides a "failureProperty" attribute naming a build property that will be set if any easyb story results in a failure. Since Gradle sees running easyb as a successful outcome and will happily keep chugging along as long as easyb ran without error, you can use the "failureProperty" in conjunction with Ant's "fail" task to fail the build if a story does not pass:

// easyb task configured with failureProperty
ant.easyb( classpath: sourceSets.test.runtimeClasspath.asPath, failureProperty:'easyb_failed' ) {
        report( location:"${project.testResultsDir}/story.txt", format:"txtstory" )
        behaviors( dir: "src/test/stories" ) {
            include( name:"**/*.story" )
        }
   }

// checking the failureProperty
ant.fail( if:'easyb_failed', message: 'Failures in easyb stories' )

The final code in the build.gradle file, with dependencies defined and easyb added to Gradle's "check" task, might end up looking something like this:

dependencies {
    testRuntime("org.easyb:easyb:0.9.5.2@jar") { transitive = true }
}

check << {

    ant.taskdef(name: "easyb", classname:"org.easyb.ant.BehaviorRunnerTask", classpath: sourceSets.test.runtimeClasspath.asPath)

    ant.easyb( classpath: sourceSets.test.runtimeClasspath.asPath, failureProperty:'easyb_failed' ) {
        report( location:"${project.testResultsDir}/story.txt", format:"txtstory" )
        behaviors( dir: "src/test/stories" ) {
            include( name:"**/*.story" )
        }
   }

    ant.fail( if:'easyb_failed', message: 'Failures in easyb stories' )
}

When the build is run (in this case, on a Groovy project), the output might look like this:

user$ gradle clean check
:clean
:compileJava
:compileGroovy
:processResources
:classes
:compileTestJava
:compileTestGroovy
:processTestResources
:testClasses
:test
:check

BUILD SUCCESSFUL

Total time: 7.267 secs

If there are problems, or if any of your stories fail and you're failing the build based on the "failureProperty", then you might see output like this:

user$ gradle clean check
:clean
:compileJava
:compileGroovy
:processResources
:classes
:compileTestJava
:compileTestGroovy
:processTestResources
:testClasses
:test
:check

FAILURE: Build failed with an exception.

* Where:
Build file '/path/to/project/build.gradle' line: 39

* What went wrong:
Execution failed for task ':check'.
Cause: Failures in easyb stories

* Try:
Run with -s or -d option to get more details. Run with -S option to get the full (very verbose) stacktrace.

BUILD FAILED

Total time: 7.443 secs

Note, if the build fails, and you haven't yet successfully run easyb in Gradle yet, you might not have it configured correctly. However, the "normal" output isn't particularly helpful in telling you what's wrong, since Gradle seems to supress most of the error output. One of the more helpful things at that point is to run your build using the "-d" option to get the full debugging output:

gradle -d clean check

With the extensive debugging output, you should be able to trace through the build and see what's going on; often, the problem is a "ClassDefNotFound" error as a result of an artifact missing from the dependencies or an error in parsing your easyb stories or something along those lines.

Writing a simple easyb plugin

NOTE: I wrote this article up, published it, then found this PluginAPI wiki article on the easyb site. The only difference I saw was the package names of the classes involved. Just in case this article provides a bit of clarification, I'll keep it around. . . .

The easyb BDD framework provides a ton of great functionality out of the box, but as with anything, sometimes there are features that you want that the developers can't predict. The easyb developers realized this and created a framework for adding plugins to extend easyb and make it do what you need it to.

Plugins for easyb implement the EasybPlugin interface, basically providing behavior before and after easyb "events" (by "events", I mean when easyb runs somethng like "given" or "scenario"). The EasybPlugin interface looks like this:

package org.easyb.plugin

interface EasybPlugin {
    String getName()
    def beforeScenario(Binding binding)
    def afterScenario(Binding binding)

    def beforeGiven(Binding binding)
    def afterGiven(Binding binding)

    def beforeWhen(Binding binding)
    def afterWhen(Binding binding)

    def beforeThen(Binding binding)
    def afterThen(Binding binding)

    def beforeStory(Binding binding)
    def afterStory(Binding binding)

    void setClassLoader(ClassLoader classLoader)
}

Most of the time, however, you probably don't need to provide actions for *every* event, so instead of implementing EasybPlugin, you can just extend BasePlugin which provides no-op implementations of the "event" methods, overriding whichever methods you need.

Let's get into an example. Let's say we want to extend easyb to mimic injecting resources into a story run with the Grails easyb plugin (see my post here for the story behind that). The first thing we want to do is set up our Groovy class and implement the one abstract method in BasePlugin, getName():

class GrailsInjectPlugin extends BasePlugin {
    public String getName() { "grails-inject" }
}

The getName() method serves two purposes: first, in the easyb story, you'll have a "using" line with the value that getName() returns, signaling easyb to use the plugin. Second, easyb will use this name to lookup the plugin via its PluginLocator class.

At this point, we could compile and include the plugin, and it would be valid, but it wouldn't actually do anything. To extend easyb's behavior, we need to override one of the "event" methods. In the case of injecting Grails resources, we want to set up our resources before an entire story executes (similar to how they'd be initialized in a Grails test, def-ing them at the beginning of a test class). We'll override the beforeStory() method:

    def beforeStory(Binding binding) {
        binding.inject = { beanName ->
            binding."${beanName}" = 
                ApplicationHolder.application.mainContext.getBean(beanName)
        }
    }

The "event" methods get passed a Binding implementation which basically is what easyb will use to look up values, methods, and the like that it encounters in its stories. In our beforeStory() implementation, we're telling the binding that it now has an "inject" method that takes a bean name as an argument. The method looks up that bean in the Grails Spring context, then, in the binding, associates the bean with the passed in bean name.

The final code for the plugin class looks like this:

package com.agileice.easyb.plugin

import org.codehaus.groovy.grails.commons.ApplicationHolder
import org.easyb.plugin.BasePlugin

class GrailsInjectPlugin extends BasePlugin {

    public String getName() { "grails-inject" }

    def beforeStory(Binding binding) {
        binding.inject = { beanName ->
            binding."${beanName}"= 
                ApplicationHolder.application.mainContext.getBean(beanName)
        }
    }
}

It's all fairly simple and straightforward . . . the easyb developers did a great job in making plugin development easy.

The next step in getting the plugin into your application is to include it in a JAR file. In addition to our plugin classes, we need to include some information in the JAR's META-INF directory. When easyb is looking for plugin implementations, it uses the sun.misc.Service class, so we need to include a file named org.easyb.plugin.EasybPlugin in the JAR's META-INF/services directory to tell the Service class that we provide an implementation of EasybPlugin. That file will contain the name of our implementation class:

com.agileice.easyb.plugin.GrailsInjectPlugin # inject Grails beans

At this point, we can compile our plugin, JAR up our classes and the META-INF information, and with the JAR on the classpath, run our easyb stories. To use the plugin in a story, we include a "using" line with the name of our plugin (the value returned by the getName() method). With the plugin included, we can use the keywords we've defined:

using "grails-inject"

inject "grailsApplication"
inject "someService"

scenario "Grails App injection", {

    given "injected Grails resources"
    then "the Grails application should not be null", {
        grailsApplication.shouldNotBe null
    }
    and "the service instance should not be null", {
        someService.shouldNotBe null
    }
}

Now, when the story runs, easyb should pick up our plugin, handle our behavior, and execute our scenarios. Note, the above examples should work with the version of easyb included with the Grails easyb plugin (marked 0.9.7). If you're working with an older version (say, from a Maven repository where the latest version is 0.9.5), you'll probably have to implement all of EasybPlugin yourself (since it looks like BasePlugin doesn't exist there), changing the "def" for each method to "void". Once you do that, everything else should work the same.

Injecting resources in easyb Grails stories

Note: the functionality described in this post and the which the plugin described below provides have been incorporated into the latest version of the easyb Grails plugin (plugin version 1.2). I'll leave this post here for historical context.

Normally, when you're writing Grails tests, you can have Grails inject things like service instances or the Grails application itself by simply def-ing them in your test case:

class MyTests {
   def grailsApplication
   def someService
}

However, when you're working with with the easyb Grails plugin, the stories and specifications don't go through the same process as the standard tests, so you can't just def something at the beginning of the story and access it in the scenarios.

Instead, you have to access things through the Grails ApplicationHolder, a class that exists only to provide access to the GrailsApplication instance. Once you have the application instance (and through that, the main Spring context), you can use the getBean() method to reference things that you'd normally have injected:

import org.codehaus.groovy.grails.commons.ApplicationHolder
. . .
ApplicationHolder.application.mainContext.getBean("someService")

Putting that all together with a story might look something like this:

import org.codehaus.groovy.grails.commons.ApplicationHolder

scenario "Grails App injection", {

   given "service instances initialized from Grails ApplicationHolder", {
       grailsApplication = ApplicationHolder.application
       someService = ApplicationHolder.application.mainContext.getBean("someService")
   }
   then "the Grails application should not be null", {
       grailsApplication.shouldNotBe null
   }
   and "the service instance should not be null", {
       someService.shouldNotBe null
   }

}

While that's not too bad, it's actually a good bit more noise than I'd like, especially since I can see doing this often in my Grails integration tests. To make it a little easier to read, I ended up writing an easyb plugin to make it look like I was actually injecting things in the easyb stories (I wrote up the process I used here . . . there's also a good wiki entry about the Plugin API at the easyb site). With the plugin I wrote (jar file available here under the MIT license, though it's subject to change as I learn more about easyb . . . the source will eventually be available if people find it useful), I can write a story with easyb's "using" syntax to include the plugin and the custom "inject" keyword:

using "grails-inject"

inject "grailsApplication"
inject "someService"

scenario "Grails App injection", {

   given "Service instances initialized from Grails ApplicationHolder"
   then "the Grails application should not be null", {
       grailsApplication.shouldNotBe null
   }
   and "the service instance should not be null", {
       someService.shouldNotBe null
   }

}

With the instances injected into my tests, I can now call whatever methods I want on them, dealing with them in the same way I would in a standard Grails test case. NOTE: if you want to use the plugin above, download it and drop it in the lib directory of your Grails app. When you run "grails easyb-test", easyb should pick it up and make the syntax available to your stories and specifications.

Getting started with the easyb plugin for BDD in Grails

One of the more exciting things I've learned about recently is doing behavior driven development (BDD) with a framework called easyb. Easyb allows you to test your system by defining behavior in stories and specifications in a way that allows your "non-developer" stakeholders to understand them relatively easily. I also find that it helps me think more clearly about what my systems should actually be doing and how the pieces should interact. Like I said, I'm really excited by it, and I really wanted to start incorporating easyb where I can. Since I've been doing a ton of Grails work lately, naturally I wanted to try to incorporate it into my Grails workflow. Fortunately, there's a plugin for that, the Grails easyb plugin.

The first step in doing BDD with easyb in Grails is, of course, to install the plugin, and you do that in the same way you install any other Grails plugin: in your project directory, issue the command "grails install-plugin easyb". After all the noise of starting Grails and downloading the plugin artifact, you'll see:

Plug-in provides the following new scripts:
------------------------------------------
grails easyb-test

Now we can mix easyb stories and specifications in with the usual Grails tests and run them using "grails easyb-test" command. Note, however, that at this point, to execute both your easyb and usual Grails tests, you have to run two commands, both "grails test-app" and "grails easyb-test": just running the normal "test-app" won't also execute the easyb tests, and running the easyb tests won't also execute the normal tests.

So let's say I'm working with an application that takes dry cleaning orders and I want to test the behavior of the service that creates orders when a customer drops of some suits to be dry cleaned. Assuming I have a class called DropOffService, I'll create a new file in the test/unit directory, calling it either DropOffService.story or DropOffServiceStory.groovy (either will work).

Now I can define the behavior I want in the story file. I won't go in-depth on the details of writing easyb stories since there's good documentation on the easyb site for that, but the short form is that I create scenarios with given conditions, some event that occurs, and a pre-defined outcome. To test our service's behavior, I might define the story like this:

scenario "customer drops off suits", {
    given "a drop-off service"
    and "a customer with suits"
    when "the customer drops off the suits"
    then "the service should generate a new order"
}

At this point, I haven't actually written any code that will exercise the service, but I can still run "grails easyb-test". When I do, the plugin looks for valid easyb stories and specifications, runs them, and generates reports in the test/reports directory. The easyb plugin generates both text and html reports just like the standard Grails tests; however, it also produces XML output, too, potentially useful for reporting in something like a continuous integration system. Unlike the standard Grails reporting, instead of getting a file for each test class, the easyb plugin generates a report for each *type* of test. So we end up with a reports listing that looks like this:

test/reports/html:
easyb-functional.html  easyb_img04.jpg         easyb_report.css
easyb_img01.gif        easyb_img05.jpg         easyb_spacer.gif
easyb_img02.jpg        easyb_img06.jpg         easyb-unit.html
easyb_img03.jpg        easyb-integration.html  prototype.js

test/reports/plain:
specifications-functional.txt   stories-functional.txt
specifications-integration.txt  stories-integration.txt
specifications-unit.txt         stories-unit.txt

test/reports/xml:
easyb-functional.xml  easyb-integration.xml  easyb-unit.xml

If we actually look at the reports, we'll see that the single scenario I've written has a result of "pending" since I haven't actually implemented any code to exercise the drop-off service. At the point where I actually *do* something in the story, the result will change depending on whether or not the scenario produces the outcome I'm expecting.

So there's the basics of using easyb with Grails. There's definitely more to explore beyond this simple example (testing controllers, using dependency injection, and such), and I'll do that in future postings.