Getting started with the easyb plugin for BDD in Grails

One of the more exciting things I've learned about recently is doing behavior driven development (BDD) with a framework called easyb. Easyb allows you to test your system by defining behavior in stories and specifications in a way that allows your "non-developer" stakeholders to understand them relatively easily. I also find that it helps me think more clearly about what my systems should actually be doing and how the pieces should interact. Like I said, I'm really excited by it, and I really wanted to start incorporating easyb where I can. Since I've been doing a ton of Grails work lately, naturally I wanted to try to incorporate it into my Grails workflow. Fortunately, there's a plugin for that, the Grails easyb plugin.

The first step in doing BDD with easyb in Grails is, of course, to install the plugin, and you do that in the same way you install any other Grails plugin: in your project directory, issue the command "grails install-plugin easyb". After all the noise of starting Grails and downloading the plugin artifact, you'll see:

Plug-in provides the following new scripts:
------------------------------------------
grails easyb-test

Now we can mix easyb stories and specifications in with the usual Grails tests and run them using "grails easyb-test" command. Note, however, that at this point, to execute both your easyb and usual Grails tests, you have to run two commands, both "grails test-app" and "grails easyb-test": just running the normal "test-app" won't also execute the easyb tests, and running the easyb tests won't also execute the normal tests.

So let's say I'm working with an application that takes dry cleaning orders and I want to test the behavior of the service that creates orders when a customer drops of some suits to be dry cleaned. Assuming I have a class called DropOffService, I'll create a new file in the test/unit directory, calling it either DropOffService.story or DropOffServiceStory.groovy (either will work).

Now I can define the behavior I want in the story file. I won't go in-depth on the details of writing easyb stories since there's good documentation on the easyb site for that, but the short form is that I create scenarios with given conditions, some event that occurs, and a pre-defined outcome. To test our service's behavior, I might define the story like this:

scenario "customer drops off suits", {
    given "a drop-off service"
    and "a customer with suits"
    when "the customer drops off the suits"
    then "the service should generate a new order"
}

At this point, I haven't actually written any code that will exercise the service, but I can still run "grails easyb-test". When I do, the plugin looks for valid easyb stories and specifications, runs them, and generates reports in the test/reports directory. The easyb plugin generates both text and html reports just like the standard Grails tests; however, it also produces XML output, too, potentially useful for reporting in something like a continuous integration system. Unlike the standard Grails reporting, instead of getting a file for each test class, the easyb plugin generates a report for each *type* of test. So we end up with a reports listing that looks like this:

test/reports/html:
easyb-functional.html  easyb_img04.jpg         easyb_report.css
easyb_img01.gif        easyb_img05.jpg         easyb_spacer.gif
easyb_img02.jpg        easyb_img06.jpg         easyb-unit.html
easyb_img03.jpg        easyb-integration.html  prototype.js

test/reports/plain:
specifications-functional.txt   stories-functional.txt
specifications-integration.txt  stories-integration.txt
specifications-unit.txt         stories-unit.txt

test/reports/xml:
easyb-functional.xml  easyb-integration.xml  easyb-unit.xml

If we actually look at the reports, we'll see that the single scenario I've written has a result of "pending" since I haven't actually implemented any code to exercise the drop-off service. At the point where I actually *do* something in the story, the result will change depending on whether or not the scenario produces the outcome I'm expecting.

So there's the basics of using easyb with Grails. There's definitely more to explore beyond this simple example (testing controllers, using dependency injection, and such), and I'll do that in future postings.

Binding the Grails DataSource in JNDI for Integration Testing

One of the components of the Grails application we're working on requires a database connection to be looked up via JNDI. That's fine for running in an application container in production where we can configure JNDI data sources easily enough, but for our integration tests using the embedded HSQLDB, we needed a way to get the Grails-configured testing data source via JNDI.

While we could have set up a source in jetty-env.xml and placed that file in WEB-INF, from a number of postings on the 'net (see, for example, this thread), it seems like the integration tests don't actually run within the web application context Jetty provides. So, instead, we decided to register the data source programmatically as we needed it in our integration tests.

Fortunately, Grails provides access easily enough to the testing data source (configured in grails-app/conf/DataSource.groovy) merely by adding a "def dataSource" in our integration test class defintion:

class MyClassTests extends GroovyTestCase {
    def dataSource // auto-wired by Grails

    . . . setup, test methods, etc . . .
}

With the data source defined, we can now access it as we need it.

The next step is to create an InitialContext and bind the data source:

def ctx = new javax.naming.InitialContext()
ctx.bind("java:/testing-db", dataSource)

Now, with the data source bound in JNDI, we can retrieve it and use it where we need it:

def ctx = new javax.naming.InitialContext()
def conn = ctx.lookup("java:/testing-db")
def sql = new groovy.sql.Sql(conn)

// create a table to hold some test data
sql.execute("create table test_data(id identity, some_column varchar, col_ts timestamp default current_timestamp)")

Of course, if we didn't want to worry about accidentally messing up the Grails testing data source, we could always have set up our own HSQLDB data source and bound it:

import org.apache.commons.dbcp.BasicDataSource

def ds = new BasicDataSource()
ds.setUrl("jdbc:hsqldb:mem:myTestDb")
ds.setUsername("sa")
ds.setPassword("")
ds.setDriverClassName("org.hsqldb.jdbcDriver")

def ctx = new InitialContext()
ctx.bind("java:/testing-db", ds)

However, in our case, using the standard Grails data source worked just fine. It's probably worth our while to try to set up a data source through jetty-env.xml for using our components requiring JNDI in the "run-app" development environment, but our integration tests are running, and that's good enough for now.

Pretty-printing XML results returned from Groovy RESTClient

In another blog post here, someone commented that they had to fall back to HTTPBuilder to output the XML returned from a REST request because RESTClient automatically parses the XML and returns a GPathResult NodeChild object. That's certainly one solution (and it gives you a good amount of control over both the request and its output), but if you *really* want to use RESTClient, it's still possible to format the results parsed via XMLSlurper.

When I needed to do this, I ended up using a combination of Groovy's StreamingMarkupBuilder to create the initial output, then I used Xalan to transform that output into easier to read XML. The method ended up looking like this:

import groovy.xml.StreamingMarkupBuilder

import javax.xml.transform.TransformerFactory
import javax.xml.transform.Transformer
import javax.xml.transform.OutputKeys
import javax.xml.transform.stream.StreamResult
import javax.xml.transform.stream.StreamSource

/**
 *  pretty prints the GPathResult NodeChild
 */
def outputFormattedXml(node) {
    def xml = new StreamingMarkupBuilder().bind {
        mkp.declareNamespace("":node.namespaceURI())
        mkp.yield(node)
    }

    def factory = TransformerFactory.newInstance()
    def transformer = factory.newTransformer()
    transformer.setOutputProperty(OutputKeys.INDENT, 'yes')

    // figured this out by looking at Xalan's serializer.jar
    // org/apache/xml/serializer/output_xml.properties
    transformer.setOutputProperty("{http\u003a//xml.apache.org/xalan}indent-amount", "2")
    def result = new StreamResult(new StringWriter())
    transformer.transform(new StreamSource(new ByteArrayInputStream(xml.toString().bytes)), result)

    return result.writer.toString()
}

As I mentioned in the code comments, there were some pieces I had to figure out from looking in Xalan's serializer.jar. For modifying other output properties, take a look at org/apache/xml/serializer/output_xml.properties in serializer.jar to see just what you can customize and how to refer to it in your code.

In the end, the above method could probably be used to pretty-print any NodeChild object from GPathResult. There may be an easier or "more Groovy" way to output this kind of thing, but this allows the use of both the easy-to-use RESTClient and the easy-to-use XmlSlurper without having to worry much about making the output human-readable.

Grails type conversion outside of domain objects

In a recent Grails project, we needed a way to do type conversion outside of merely marshalling values for a domain object (i.e. outside of, say, automatically loading up the properties of a domain object from request parameters, a job which Grails does really well). Specifically, we needed to convert the String representation of a Date into an actual Date object for use in our domain object's findByLastUpdatedGreaterThan() method. Initially, we hoped the method would automatically convert the type for us, but instead it threw exceptions saying "No signature of method . . . is applicable for argument types" because it couldn't handle a String instead of a Date.

When Grails converts types for marshalling request parameters to domain objects, behind the scenes, it uses a GrailsDataBinder instance, converting values in the process of loading up the object's properties. And while the GrailsDataBinder implements Spring's TypeConverter and gives us access to the convertIfNecessary() method, it also requires a domain object instance in its constructor so it has a target to bind values to. Since all we were doing is simple type conversion, we wanted a different way.

Enter Spring's SimpleTypeConverter, a class that really only does one thing: convert from a String type to an instance of a target class. SimpleTypeConverter allows us to register custom property editors, the backbone of the type conversion process, so the first task was to implement a PropertyEditor (extending PropertyEditorSupport) that would let us take a series of date formats and parse them to a Date. Thanks to some very detailed information in a response to this StackOverflow Question, implementing our CustomDateEditor was fairly easy:

import java.beans.PropertyEditorSupport
import java.text.SimpleDateFormat
import java.text.ParseException

/**
 * A property editor that allows for handling many different
 * date formats at once (as opposed to Spring's default CustomDateEditor
 * which only handles one date format configured at instantiation)
 *
 * This class is modified from information found at:
 * http://stackoverflow.com/questions/963922/grails-date-unmarshalling
 */
class CustomDateEditor extends PropertyEditorSupport {

  private final List< String> formats

  CustomDateEditor() {
    // in our production code, we get the formats from our Grails config
    def formats = [ "yyyyMMdd-HHmmss", "yyyy-mm-dd HH:mm:ss.S" ]
    List< String> formatList = new ArrayList< String>(formats.size())
    formats.each { format ->
      formatList << format.toString() // Force String values (eg. for GStrings)
    }
    this.formats = Collections.unmodifiableList(formatList)
  }

  def String getAsText() {
    return this.value.toString()
  }

  def void setAsText(String s) throws IllegalArgumentException {
    if (s) {
      formats.each { format ->
        def df = new SimpleDateFormat(format)
        try {
          this.value = df.parse(s)
          return
        } catch (ParseException e) {
          // Ignore
        }
      }
    }
  }
}

Next, we wanted to make this custom Date editor available across the application, so we added it to our resources.groovy Spring config:

beans = {
    customDateEditor(CustomDateEditor) { bean ->
        bean.singleton = false // editors store state so are not thread-safe
    }
}

Now anywhere we wanted to do type conversion, all our classes needed to do was create a new SimpleTypeConverter, get a CustomDateEditor instance from Spring, and register that editor with the SimpleTypeConverter:

// grailsApplication auto-wired by Grails
def editor = grailsApplication.mainContext.getBean("customDateEditor") 
def converter = new SimpleTypeConverter()
converter.registerCustomEditor( Date.class, editor )

With the converter set up and our custom Date editor registered, it was now a simple matter of doing the conversion:

// NOTE: production code determines type dynamically from field being queried
def type = Date.class
def converted = converter.convertIfNecessary( value, type )

If we had wanted to make our custom editor available for marshalling values to a domain object, we could have easily created an PropertyEditorRegistrar object and configured it via Spring so Grails would automatically use our property editor (as described in the StackOverflow Question above). But, again, all we wanted was fairly straightforward type conversion outside of a domain object. The combination of Spring's SimpleTypeConverter and our CustomDateEditor gave us that.

Grails integration testing: GroovyTestCase v. GrailsUnitTestCase

Working on a Grails project in IntelliJ over the last few weeks, I've accumulated a number of unit and integration tests for various classes. Up until now, since I've only been working on isolated sections, I've been running the test classes individually, but I finally did a refactor that cut across a number of areas that really required me to run most of the tests. Since I was feeling a bit fuzzy headed with a late-summer cold, I started running the tests one by one, each one passing, but I quickly realized that was going to be a major pain.

Since Grails allows you to run all the tests at once, I figured I'd just run "grails test-app" and be done with it, but to my surprise, a number of tests which had passed when I ran them individually failed when I ran the tests in one batch. Looking more closely and doing a bit more logging, it turned out that my controller classes that did content negotiation and used a "withFormat" block to check for XML content weren't handling the XML content: they were either running the html/form section rather than the section for handling xml content or they were running nothing at all.

After a couple hours of head scratching and trying different approaches, it looked like the Grails configuration properties (which contain pieces telling Grails how to deal with Accept headers and different MIME types) weren't being made available to test cases in the integration tests. In a bit of desperation, I basically put the configuration parameters for doing content negotiation in one of my controller integration test case's setUp() methods:

def grailsApplication
void setUp() {
    super.setUp()
    grailsApplication.config.grails.mime.file.extensions = true
    grailsApplication.config.grails.mime.use.accept.header = true
    grailsApplication.config.grails.mime.types = [ html: ['text/html','application/xhtml+xml'],
                      xml: ['text/xml', 'application/xml'],
                      text: 'text/plain',
                      js: 'text/javascript',
                      rss: 'application/rss+xml',
                      atom: 'application/atom+xml',
                      css: 'text/css',
                      csv: 'text/csv',
                      all: '*/*',
                      json: ['application/json','text/json'],
                      form: 'application/x-www-form-urlencoded',
                      multipartForm: 'multipart/form-data'
                    ]
}

Magically, it worked. And not only that, it magically also made the other controller tests work, too, even without editing their setUp() methods. It seemed something was bleeding over and/or not getting cleaned up somewhere.

I mentioned all this chatting with my friend Eric (see his blog at Sword Systems for some really great insights on Java, Groovy, and tools of the trade) who proceeded to tell me that he discovered some weirdness with GrailsUnitTestCase and how it mucked with the environment. I looked at the test cases I had (mostly generated automatically by Grails/IntelliJ), and they were a mix of GrailsUnitTestCase (integration tests for domain classes) and GroovyTestCase (integration tests for controllers, the failing tests). It turns out that the GroovyTestCase tests would work just fine on their own, but when mixed in the same environment with the GrailsUnitTestCase, somehow they lost their access to the Grails config.

Doing a bit more research, I found this Grails bug report, which mentions that "GrailsUnitTestCase sets grailsApplication.config to null in its tearDown routine which causes any additional tests . . ." to have problems. It seems even with this known problem, Grails still generates integration tests extending from GrailsUnitTestCase.

So the solution was to use GrailsUnitTestCase (or its derivatives like ControllerUnitTestCase) *only* for unit tests (in spite of the examples at pages like Grails - Testing Controllers); for integration tests, use GroovyTestCase (like the examples in the main Grails documentation on testing). Once I changed all my integration tests to GroovyTestCase and ran them all as a batch, they all passed. I took that ugly configuration code out of the setUp() method and ran it again, and they all passed. Problem solved (thanks, Eric!).