Binding the Grails DataSource in JNDI for Integration Testing

One of the components of the Grails application we're working on requires a database connection to be looked up via JNDI. That's fine for running in an application container in production where we can configure JNDI data sources easily enough, but for our integration tests using the embedded HSQLDB, we needed a way to get the Grails-configured testing data source via JNDI.

While we could have set up a source in jetty-env.xml and placed that file in WEB-INF, from a number of postings on the 'net (see, for example, this thread), it seems like the integration tests don't actually run within the web application context Jetty provides. So, instead, we decided to register the data source programmatically as we needed it in our integration tests.

Fortunately, Grails provides access easily enough to the testing data source (configured in grails-app/conf/DataSource.groovy) merely by adding a "def dataSource" in our integration test class defintion:

class MyClassTests extends GroovyTestCase {
    def dataSource // auto-wired by Grails

    . . . setup, test methods, etc . . .
}

With the data source defined, we can now access it as we need it.

The next step is to create an InitialContext and bind the data source:

def ctx = new javax.naming.InitialContext()
ctx.bind("java:/testing-db", dataSource)

Now, with the data source bound in JNDI, we can retrieve it and use it where we need it:

def ctx = new javax.naming.InitialContext()
def conn = ctx.lookup("java:/testing-db")
def sql = new groovy.sql.Sql(conn)

// create a table to hold some test data
sql.execute("create table test_data(id identity, some_column varchar, col_ts timestamp default current_timestamp)")

Of course, if we didn't want to worry about accidentally messing up the Grails testing data source, we could always have set up our own HSQLDB data source and bound it:

import org.apache.commons.dbcp.BasicDataSource

def ds = new BasicDataSource()
ds.setUrl("jdbc:hsqldb:mem:myTestDb")
ds.setUsername("sa")
ds.setPassword("")
ds.setDriverClassName("org.hsqldb.jdbcDriver")

def ctx = new InitialContext()
ctx.bind("java:/testing-db", ds)

However, in our case, using the standard Grails data source worked just fine. It's probably worth our while to try to set up a data source through jetty-env.xml for using our components requiring JNDI in the "run-app" development environment, but our integration tests are running, and that's good enough for now.

Pretty-printing XML results returned from Groovy RESTClient

In another blog post here, someone commented that they had to fall back to HTTPBuilder to output the XML returned from a REST request because RESTClient automatically parses the XML and returns a GPathResult NodeChild object. That's certainly one solution (and it gives you a good amount of control over both the request and its output), but if you *really* want to use RESTClient, it's still possible to format the results parsed via XMLSlurper.

When I needed to do this, I ended up using a combination of Groovy's StreamingMarkupBuilder to create the initial output, then I used Xalan to transform that output into easier to read XML. The method ended up looking like this:

import groovy.xml.StreamingMarkupBuilder

import javax.xml.transform.TransformerFactory
import javax.xml.transform.Transformer
import javax.xml.transform.OutputKeys
import javax.xml.transform.stream.StreamResult
import javax.xml.transform.stream.StreamSource

/**
 *  pretty prints the GPathResult NodeChild
 */
def outputFormattedXml(node) {
    def xml = new StreamingMarkupBuilder().bind {
        mkp.declareNamespace("":node.namespaceURI())
        mkp.yield(node)
    }

    def factory = TransformerFactory.newInstance()
    def transformer = factory.newTransformer()
    transformer.setOutputProperty(OutputKeys.INDENT, 'yes')

    // figured this out by looking at Xalan's serializer.jar
    // org/apache/xml/serializer/output_xml.properties
    transformer.setOutputProperty("{http\u003a//xml.apache.org/xalan}indent-amount", "2")
    def result = new StreamResult(new StringWriter())
    transformer.transform(new StreamSource(new ByteArrayInputStream(xml.toString().bytes)), result)

    return result.writer.toString()
}

As I mentioned in the code comments, there were some pieces I had to figure out from looking in Xalan's serializer.jar. For modifying other output properties, take a look at org/apache/xml/serializer/output_xml.properties in serializer.jar to see just what you can customize and how to refer to it in your code.

In the end, the above method could probably be used to pretty-print any NodeChild object from GPathResult. There may be an easier or "more Groovy" way to output this kind of thing, but this allows the use of both the easy-to-use RESTClient and the easy-to-use XmlSlurper without having to worry much about making the output human-readable.

Grails type conversion outside of domain objects

In a recent Grails project, we needed a way to do type conversion outside of merely marshalling values for a domain object (i.e. outside of, say, automatically loading up the properties of a domain object from request parameters, a job which Grails does really well). Specifically, we needed to convert the String representation of a Date into an actual Date object for use in our domain object's findByLastUpdatedGreaterThan() method. Initially, we hoped the method would automatically convert the type for us, but instead it threw exceptions saying "No signature of method . . . is applicable for argument types" because it couldn't handle a String instead of a Date.

When Grails converts types for marshalling request parameters to domain objects, behind the scenes, it uses a GrailsDataBinder instance, converting values in the process of loading up the object's properties. And while the GrailsDataBinder implements Spring's TypeConverter and gives us access to the convertIfNecessary() method, it also requires a domain object instance in its constructor so it has a target to bind values to. Since all we were doing is simple type conversion, we wanted a different way.

Enter Spring's SimpleTypeConverter, a class that really only does one thing: convert from a String type to an instance of a target class. SimpleTypeConverter allows us to register custom property editors, the backbone of the type conversion process, so the first task was to implement a PropertyEditor (extending PropertyEditorSupport) that would let us take a series of date formats and parse them to a Date. Thanks to some very detailed information in a response to this StackOverflow Question, implementing our CustomDateEditor was fairly easy:

import java.beans.PropertyEditorSupport
import java.text.SimpleDateFormat
import java.text.ParseException

/**
 * A property editor that allows for handling many different
 * date formats at once (as opposed to Spring's default CustomDateEditor
 * which only handles one date format configured at instantiation)
 *
 * This class is modified from information found at:
 * http://stackoverflow.com/questions/963922/grails-date-unmarshalling
 */
class CustomDateEditor extends PropertyEditorSupport {

  private final List< String> formats

  CustomDateEditor() {
    // in our production code, we get the formats from our Grails config
    def formats = [ "yyyyMMdd-HHmmss", "yyyy-mm-dd HH:mm:ss.S" ]
    List< String> formatList = new ArrayList< String>(formats.size())
    formats.each { format ->
      formatList << format.toString() // Force String values (eg. for GStrings)
    }
    this.formats = Collections.unmodifiableList(formatList)
  }

  def String getAsText() {
    return this.value.toString()
  }

  def void setAsText(String s) throws IllegalArgumentException {
    if (s) {
      formats.each { format ->
        def df = new SimpleDateFormat(format)
        try {
          this.value = df.parse(s)
          return
        } catch (ParseException e) {
          // Ignore
        }
      }
    }
  }
}

Next, we wanted to make this custom Date editor available across the application, so we added it to our resources.groovy Spring config:

beans = {
    customDateEditor(CustomDateEditor) { bean ->
        bean.singleton = false // editors store state so are not thread-safe
    }
}

Now anywhere we wanted to do type conversion, all our classes needed to do was create a new SimpleTypeConverter, get a CustomDateEditor instance from Spring, and register that editor with the SimpleTypeConverter:

// grailsApplication auto-wired by Grails
def editor = grailsApplication.mainContext.getBean("customDateEditor") 
def converter = new SimpleTypeConverter()
converter.registerCustomEditor( Date.class, editor )

With the converter set up and our custom Date editor registered, it was now a simple matter of doing the conversion:

// NOTE: production code determines type dynamically from field being queried
def type = Date.class
def converted = converter.convertIfNecessary( value, type )

If we had wanted to make our custom editor available for marshalling values to a domain object, we could have easily created an PropertyEditorRegistrar object and configured it via Spring so Grails would automatically use our property editor (as described in the StackOverflow Question above). But, again, all we wanted was fairly straightforward type conversion outside of a domain object. The combination of Spring's SimpleTypeConverter and our CustomDateEditor gave us that.

Grails integration testing: GroovyTestCase v. GrailsUnitTestCase

Working on a Grails project in IntelliJ over the last few weeks, I've accumulated a number of unit and integration tests for various classes. Up until now, since I've only been working on isolated sections, I've been running the test classes individually, but I finally did a refactor that cut across a number of areas that really required me to run most of the tests. Since I was feeling a bit fuzzy headed with a late-summer cold, I started running the tests one by one, each one passing, but I quickly realized that was going to be a major pain.

Since Grails allows you to run all the tests at once, I figured I'd just run "grails test-app" and be done with it, but to my surprise, a number of tests which had passed when I ran them individually failed when I ran the tests in one batch. Looking more closely and doing a bit more logging, it turned out that my controller classes that did content negotiation and used a "withFormat" block to check for XML content weren't handling the XML content: they were either running the html/form section rather than the section for handling xml content or they were running nothing at all.

After a couple hours of head scratching and trying different approaches, it looked like the Grails configuration properties (which contain pieces telling Grails how to deal with Accept headers and different MIME types) weren't being made available to test cases in the integration tests. In a bit of desperation, I basically put the configuration parameters for doing content negotiation in one of my controller integration test case's setUp() methods:

def grailsApplication
void setUp() {
    super.setUp()
    grailsApplication.config.grails.mime.file.extensions = true
    grailsApplication.config.grails.mime.use.accept.header = true
    grailsApplication.config.grails.mime.types = [ html: ['text/html','application/xhtml+xml'],
                      xml: ['text/xml', 'application/xml'],
                      text: 'text/plain',
                      js: 'text/javascript',
                      rss: 'application/rss+xml',
                      atom: 'application/atom+xml',
                      css: 'text/css',
                      csv: 'text/csv',
                      all: '*/*',
                      json: ['application/json','text/json'],
                      form: 'application/x-www-form-urlencoded',
                      multipartForm: 'multipart/form-data'
                    ]
}

Magically, it worked. And not only that, it magically also made the other controller tests work, too, even without editing their setUp() methods. It seemed something was bleeding over and/or not getting cleaned up somewhere.

I mentioned all this chatting with my friend Eric (see his blog at Sword Systems for some really great insights on Java, Groovy, and tools of the trade) who proceeded to tell me that he discovered some weirdness with GrailsUnitTestCase and how it mucked with the environment. I looked at the test cases I had (mostly generated automatically by Grails/IntelliJ), and they were a mix of GrailsUnitTestCase (integration tests for domain classes) and GroovyTestCase (integration tests for controllers, the failing tests). It turns out that the GroovyTestCase tests would work just fine on their own, but when mixed in the same environment with the GrailsUnitTestCase, somehow they lost their access to the Grails config.

Doing a bit more research, I found this Grails bug report, which mentions that "GrailsUnitTestCase sets grailsApplication.config to null in its tearDown routine which causes any additional tests . . ." to have problems. It seems even with this known problem, Grails still generates integration tests extending from GrailsUnitTestCase.

So the solution was to use GrailsUnitTestCase (or its derivatives like ControllerUnitTestCase) *only* for unit tests (in spite of the examples at pages like Grails - Testing Controllers); for integration tests, use GroovyTestCase (like the examples in the main Grails documentation on testing). Once I changed all my integration tests to GroovyTestCase and ran them all as a batch, they all passed. I took that ugly configuration code out of the setUp() method and ran it again, and they all passed. Problem solved (thanks, Eric!).