Grails integration testing: GroovyTestCase v. GrailsUnitTestCase

Working on a Grails project in IntelliJ over the last few weeks, I've accumulated a number of unit and integration tests for various classes. Up until now, since I've only been working on isolated sections, I've been running the test classes individually, but I finally did a refactor that cut across a number of areas that really required me to run most of the tests. Since I was feeling a bit fuzzy headed with a late-summer cold, I started running the tests one by one, each one passing, but I quickly realized that was going to be a major pain.

Since Grails allows you to run all the tests at once, I figured I'd just run "grails test-app" and be done with it, but to my surprise, a number of tests which had passed when I ran them individually failed when I ran the tests in one batch. Looking more closely and doing a bit more logging, it turned out that my controller classes that did content negotiation and used a "withFormat" block to check for XML content weren't handling the XML content: they were either running the html/form section rather than the section for handling xml content or they were running nothing at all.

After a couple hours of head scratching and trying different approaches, it looked like the Grails configuration properties (which contain pieces telling Grails how to deal with Accept headers and different MIME types) weren't being made available to test cases in the integration tests. In a bit of desperation, I basically put the configuration parameters for doing content negotiation in one of my controller integration test case's setUp() methods:

def grailsApplication
void setUp() {
    super.setUp()
    grailsApplication.config.grails.mime.file.extensions = true
    grailsApplication.config.grails.mime.use.accept.header = true
    grailsApplication.config.grails.mime.types = [ html: ['text/html','application/xhtml+xml'],
                      xml: ['text/xml', 'application/xml'],
                      text: 'text/plain',
                      js: 'text/javascript',
                      rss: 'application/rss+xml',
                      atom: 'application/atom+xml',
                      css: 'text/css',
                      csv: 'text/csv',
                      all: '*/*',
                      json: ['application/json','text/json'],
                      form: 'application/x-www-form-urlencoded',
                      multipartForm: 'multipart/form-data'
                    ]
}

Magically, it worked. And not only that, it magically also made the other controller tests work, too, even without editing their setUp() methods. It seemed something was bleeding over and/or not getting cleaned up somewhere.

I mentioned all this chatting with my friend Eric (see his blog at Sword Systems for some really great insights on Java, Groovy, and tools of the trade) who proceeded to tell me that he discovered some weirdness with GrailsUnitTestCase and how it mucked with the environment. I looked at the test cases I had (mostly generated automatically by Grails/IntelliJ), and they were a mix of GrailsUnitTestCase (integration tests for domain classes) and GroovyTestCase (integration tests for controllers, the failing tests). It turns out that the GroovyTestCase tests would work just fine on their own, but when mixed in the same environment with the GrailsUnitTestCase, somehow they lost their access to the Grails config.

Doing a bit more research, I found this Grails bug report, which mentions that "GrailsUnitTestCase sets grailsApplication.config to null in its tearDown routine which causes any additional tests . . ." to have problems. It seems even with this known problem, Grails still generates integration tests extending from GrailsUnitTestCase.

So the solution was to use GrailsUnitTestCase (or its derivatives like ControllerUnitTestCase) *only* for unit tests (in spite of the examples at pages like Grails - Testing Controllers); for integration tests, use GroovyTestCase (like the examples in the main Grails documentation on testing). Once I changed all my integration tests to GroovyTestCase and ran them all as a batch, they all passed. I took that ugly configuration code out of the setUp() method and ran it again, and they all passed. Problem solved (thanks, Eric!).