Coding around Shiro security in Grails integration tests

A number of our controllers work directly with the Shiro-authenticated user in a session, so when it comes to integration testing our controllers, we need a way of faking the Shiro actions. Groovy's metaprogramming techniques make it fairly easy.

There are three main pieces to be aware of in the Shiro authentication workflow: a Subject, the SecurityManager, and SecurityUtils. Creating a fake Subject is easy; the two things in Subject we need to worry about are the "principal" (more or less the user name) and whether or not that user is authenticated. We'll crate a Map with some simple closures and cast it as a Subject we can use:

import org.apache.shiro.subject.Subject

def subject = [ getPrincipal: { "iamauser" },
                isAuthenticated: { true }
              ] as Subject

Now we can use that Subject wherever we need to return a subject for Shiro to act on.

The next step is making sure we have a SecurityManager in our Shiro ThreadContext object. With the SecurityManager, the only thing we need to worry about is that it returns our subject, so we'll again fake that with a Map and the "as" keyword:

import org.apache.shiro.util.ThreadContext

ThreadContext.put( ThreadContext.SECURITY_MANAGER_KEY, 
                    [ getSubject: { subject } ] as SecurityManager )

Finally, since using Shiro's SecurityUtils makes it easy to deal with security objects in our controller code, we'll want to make sure it returns the values we want. In our case, all we really needed was the subject:

import org.apache.shiro.SecurityUtils

SecurityUtils.metaClass.static.getSubject = { subject }

That's it! In our controller test cases, we put that in our setUp() method, and we were then able to test our controllers that used Shiro's SecurityUtils to grab the current user like this:

def user
def subject = SecurityUtils.subject
if (subject.authenticated) {
  user = ShiroUser.findByUsername( subject.principal )
} else {
    // do something different
}

Putting together the pieces of the mock Shiro code looks like this:

import org.apache.shiro.subject.Subject
import org.apache.shiro.util.ThreadContext
import org.apache.shiro.SecurityUtils

def subject = [ getPrincipal: { "iamauser" },
                isAuthenticated: { true }
              ] as Subject

ThreadContext.put( ThreadContext.SECURITY_MANAGER_KEY, 
                    [ getSubject: { subject } ] as SecurityManager )

SecurityUtils.metaClass.static.getSubject = { subject }

Running easyb stories in a Gradle build

Recently, I've felt like I've been fighting Maven for all but the simplest builds, so I've been exploring alternative build systems, notably Gradle. One thing, too, that's become really important to my build processes lately is running easyb tests, so I wanted to figure out how to use easyb in my Gradle builds.

One of the nice things about Gradle is it makes good use of existing Ant tasks (why reinvent the wheel?), so integrating easyb is not difficult at all. The first step is adding the easyb artifact to the project dependencies. Unless your project is actually making use of easyb internally, you'll likely want to include it as a "testRuntime" dependency. The easyb artifact actually requires commons-cli, too, so you'll have to do one of two things:

• If you have your Gradle build set up to resolve transitive dependences with a Maven pom or Ivy file, you can set up your dependency definition to resolve transitive dependencies.


• If you're including the easyb jar directly via some other means, you can add a second "testRuntime" dependency for commons-cli.

The dependencies section might end up looking something like this:

// Access via Maven repo, 0.9.6 is currently the latest available
testRuntime("org.easyb:easyb:0.9.6@jar") { transitive = true }

// Access via local artifacts
testRuntime "org.easyb:easyb:0.9.7@jar"
testRuntime "commons-cli:commons-cli:1.2@jar"

With the dependencies in place, it's next a matter of configuring easyb to run in a Gradle task. Again, there are two options here:

• You can set up a separate task to run your easyb stories if you want to keep them separate.


• You can add the easyb run to an existing task like "check" if you know you always want to run your easyb stories whenever other tests are run.

In either case, you first need to add the easyb task via Ant's "taskdef":

ant.taskdef(name: "easyb", classname:"org.easyb.ant.BehaviorRunnerTask", 
                classpath: sourceSets.test.runtimeClasspath.asPath)

Next, you set up the easyb Ant task, telling it the classpath to use, where to put the reports and how to format them, along with where to find the behaviors you want to run:

ant.easyb( classpath: sourceSets.test.runtimeClasspath.asPath ) {
        // assume testResultsDir has already been created
        report( location:"${project.testResultsDir}/story.txt", format:"txtstory" )
        behaviors( dir: "src/test/stories" ) {
            include( name:"**/*.story" )
        }
   }

The "format" attribute can have a value of "html", "xml", "txtstory", or "txtspecification". For more information about those output formats, see the easyb documentation. Note, at this time, it doesn't appear possible to output multiple formats at once; if you require multiple formats, one workaround, depending on how extensive your tests are, might be to run easyb multiple times, once for each output format you want.

Finally, the easyb Ant task provides a "failureProperty" attribute naming a build property that will be set if any easyb story results in a failure. Since Gradle sees running easyb as a successful outcome and will happily keep chugging along as long as easyb ran without error, you can use the "failureProperty" in conjunction with Ant's "fail" task to fail the build if a story does not pass:

// easyb task configured with failureProperty
ant.easyb( classpath: sourceSets.test.runtimeClasspath.asPath, failureProperty:'easyb_failed' ) {
        report( location:"${project.testResultsDir}/story.txt", format:"txtstory" )
        behaviors( dir: "src/test/stories" ) {
            include( name:"**/*.story" )
        }
   }

// checking the failureProperty
ant.fail( if:'easyb_failed', message: 'Failures in easyb stories' )

The final code in the build.gradle file, with dependencies defined and easyb added to Gradle's "check" task, might end up looking something like this:

dependencies {
    testRuntime("org.easyb:easyb:0.9.5.2@jar") { transitive = true }
}

check << {

    ant.taskdef(name: "easyb", classname:"org.easyb.ant.BehaviorRunnerTask", classpath: sourceSets.test.runtimeClasspath.asPath)

    ant.easyb( classpath: sourceSets.test.runtimeClasspath.asPath, failureProperty:'easyb_failed' ) {
        report( location:"${project.testResultsDir}/story.txt", format:"txtstory" )
        behaviors( dir: "src/test/stories" ) {
            include( name:"**/*.story" )
        }
   }

    ant.fail( if:'easyb_failed', message: 'Failures in easyb stories' )
}

When the build is run (in this case, on a Groovy project), the output might look like this:

user$ gradle clean check
:clean
:compileJava
:compileGroovy
:processResources
:classes
:compileTestJava
:compileTestGroovy
:processTestResources
:testClasses
:test
:check

BUILD SUCCESSFUL

Total time: 7.267 secs

If there are problems, or if any of your stories fail and you're failing the build based on the "failureProperty", then you might see output like this:

user$ gradle clean check
:clean
:compileJava
:compileGroovy
:processResources
:classes
:compileTestJava
:compileTestGroovy
:processTestResources
:testClasses
:test
:check

FAILURE: Build failed with an exception.

* Where:
Build file '/path/to/project/build.gradle' line: 39

* What went wrong:
Execution failed for task ':check'.
Cause: Failures in easyb stories

* Try:
Run with -s or -d option to get more details. Run with -S option to get the full (very verbose) stacktrace.

BUILD FAILED

Total time: 7.443 secs

Note, if the build fails, and you haven't yet successfully run easyb in Gradle yet, you might not have it configured correctly. However, the "normal" output isn't particularly helpful in telling you what's wrong, since Gradle seems to supress most of the error output. One of the more helpful things at that point is to run your build using the "-d" option to get the full debugging output:

gradle -d clean check

With the extensive debugging output, you should be able to trace through the build and see what's going on; often, the problem is a "ClassDefNotFound" error as a result of an artifact missing from the dependencies or an error in parsing your easyb stories or something along those lines.

Writing a simple easyb plugin

NOTE: I wrote this article up, published it, then found this PluginAPI wiki article on the easyb site. The only difference I saw was the package names of the classes involved. Just in case this article provides a bit of clarification, I'll keep it around. . . .

The easyb BDD framework provides a ton of great functionality out of the box, but as with anything, sometimes there are features that you want that the developers can't predict. The easyb developers realized this and created a framework for adding plugins to extend easyb and make it do what you need it to.

Plugins for easyb implement the EasybPlugin interface, basically providing behavior before and after easyb "events" (by "events", I mean when easyb runs somethng like "given" or "scenario"). The EasybPlugin interface looks like this:

package org.easyb.plugin

interface EasybPlugin {
    String getName()
    def beforeScenario(Binding binding)
    def afterScenario(Binding binding)

    def beforeGiven(Binding binding)
    def afterGiven(Binding binding)

    def beforeWhen(Binding binding)
    def afterWhen(Binding binding)

    def beforeThen(Binding binding)
    def afterThen(Binding binding)

    def beforeStory(Binding binding)
    def afterStory(Binding binding)

    void setClassLoader(ClassLoader classLoader)
}

Most of the time, however, you probably don't need to provide actions for *every* event, so instead of implementing EasybPlugin, you can just extend BasePlugin which provides no-op implementations of the "event" methods, overriding whichever methods you need.

Let's get into an example. Let's say we want to extend easyb to mimic injecting resources into a story run with the Grails easyb plugin (see my post here for the story behind that). The first thing we want to do is set up our Groovy class and implement the one abstract method in BasePlugin, getName():

class GrailsInjectPlugin extends BasePlugin {
    public String getName() { "grails-inject" }
}

The getName() method serves two purposes: first, in the easyb story, you'll have a "using" line with the value that getName() returns, signaling easyb to use the plugin. Second, easyb will use this name to lookup the plugin via its PluginLocator class.

At this point, we could compile and include the plugin, and it would be valid, but it wouldn't actually do anything. To extend easyb's behavior, we need to override one of the "event" methods. In the case of injecting Grails resources, we want to set up our resources before an entire story executes (similar to how they'd be initialized in a Grails test, def-ing them at the beginning of a test class). We'll override the beforeStory() method:

    def beforeStory(Binding binding) {
        binding.inject = { beanName ->
            binding."${beanName}" = 
                ApplicationHolder.application.mainContext.getBean(beanName)
        }
    }

The "event" methods get passed a Binding implementation which basically is what easyb will use to look up values, methods, and the like that it encounters in its stories. In our beforeStory() implementation, we're telling the binding that it now has an "inject" method that takes a bean name as an argument. The method looks up that bean in the Grails Spring context, then, in the binding, associates the bean with the passed in bean name.

The final code for the plugin class looks like this:

package com.agileice.easyb.plugin

import org.codehaus.groovy.grails.commons.ApplicationHolder
import org.easyb.plugin.BasePlugin

class GrailsInjectPlugin extends BasePlugin {

    public String getName() { "grails-inject" }

    def beforeStory(Binding binding) {
        binding.inject = { beanName ->
            binding."${beanName}"= 
                ApplicationHolder.application.mainContext.getBean(beanName)
        }
    }
}

It's all fairly simple and straightforward . . . the easyb developers did a great job in making plugin development easy.

The next step in getting the plugin into your application is to include it in a JAR file. In addition to our plugin classes, we need to include some information in the JAR's META-INF directory. When easyb is looking for plugin implementations, it uses the sun.misc.Service class, so we need to include a file named org.easyb.plugin.EasybPlugin in the JAR's META-INF/services directory to tell the Service class that we provide an implementation of EasybPlugin. That file will contain the name of our implementation class:

com.agileice.easyb.plugin.GrailsInjectPlugin # inject Grails beans

At this point, we can compile our plugin, JAR up our classes and the META-INF information, and with the JAR on the classpath, run our easyb stories. To use the plugin in a story, we include a "using" line with the name of our plugin (the value returned by the getName() method). With the plugin included, we can use the keywords we've defined:

using "grails-inject"

inject "grailsApplication"
inject "someService"

scenario "Grails App injection", {

    given "injected Grails resources"
    then "the Grails application should not be null", {
        grailsApplication.shouldNotBe null
    }
    and "the service instance should not be null", {
        someService.shouldNotBe null
    }
}

Now, when the story runs, easyb should pick up our plugin, handle our behavior, and execute our scenarios. Note, the above examples should work with the version of easyb included with the Grails easyb plugin (marked 0.9.7). If you're working with an older version (say, from a Maven repository where the latest version is 0.9.5), you'll probably have to implement all of EasybPlugin yourself (since it looks like BasePlugin doesn't exist there), changing the "def" for each method to "void". Once you do that, everything else should work the same.

Injecting resources in easyb Grails stories

Note: the functionality described in this post and the which the plugin described below provides have been incorporated into the latest version of the easyb Grails plugin (plugin version 1.2). I'll leave this post here for historical context.

Normally, when you're writing Grails tests, you can have Grails inject things like service instances or the Grails application itself by simply def-ing them in your test case:

class MyTests {
   def grailsApplication
   def someService
}

However, when you're working with with the easyb Grails plugin, the stories and specifications don't go through the same process as the standard tests, so you can't just def something at the beginning of the story and access it in the scenarios.

Instead, you have to access things through the Grails ApplicationHolder, a class that exists only to provide access to the GrailsApplication instance. Once you have the application instance (and through that, the main Spring context), you can use the getBean() method to reference things that you'd normally have injected:

import org.codehaus.groovy.grails.commons.ApplicationHolder
. . .
ApplicationHolder.application.mainContext.getBean("someService")

Putting that all together with a story might look something like this:

import org.codehaus.groovy.grails.commons.ApplicationHolder

scenario "Grails App injection", {

   given "service instances initialized from Grails ApplicationHolder", {
       grailsApplication = ApplicationHolder.application
       someService = ApplicationHolder.application.mainContext.getBean("someService")
   }
   then "the Grails application should not be null", {
       grailsApplication.shouldNotBe null
   }
   and "the service instance should not be null", {
       someService.shouldNotBe null
   }

}

While that's not too bad, it's actually a good bit more noise than I'd like, especially since I can see doing this often in my Grails integration tests. To make it a little easier to read, I ended up writing an easyb plugin to make it look like I was actually injecting things in the easyb stories (I wrote up the process I used here . . . there's also a good wiki entry about the Plugin API at the easyb site). With the plugin I wrote (jar file available here under the MIT license, though it's subject to change as I learn more about easyb . . . the source will eventually be available if people find it useful), I can write a story with easyb's "using" syntax to include the plugin and the custom "inject" keyword:

using "grails-inject"

inject "grailsApplication"
inject "someService"

scenario "Grails App injection", {

   given "Service instances initialized from Grails ApplicationHolder"
   then "the Grails application should not be null", {
       grailsApplication.shouldNotBe null
   }
   and "the service instance should not be null", {
       someService.shouldNotBe null
   }

}

With the instances injected into my tests, I can now call whatever methods I want on them, dealing with them in the same way I would in a standard Grails test case. NOTE: if you want to use the plugin above, download it and drop it in the lib directory of your Grails app. When you run "grails easyb-test", easyb should pick it up and make the syntax available to your stories and specifications.

Getting started with the easyb plugin for BDD in Grails

One of the more exciting things I've learned about recently is doing behavior driven development (BDD) with a framework called easyb. Easyb allows you to test your system by defining behavior in stories and specifications in a way that allows your "non-developer" stakeholders to understand them relatively easily. I also find that it helps me think more clearly about what my systems should actually be doing and how the pieces should interact. Like I said, I'm really excited by it, and I really wanted to start incorporating easyb where I can. Since I've been doing a ton of Grails work lately, naturally I wanted to try to incorporate it into my Grails workflow. Fortunately, there's a plugin for that, the Grails easyb plugin.

The first step in doing BDD with easyb in Grails is, of course, to install the plugin, and you do that in the same way you install any other Grails plugin: in your project directory, issue the command "grails install-plugin easyb". After all the noise of starting Grails and downloading the plugin artifact, you'll see:

Plug-in provides the following new scripts:
------------------------------------------
grails easyb-test

Now we can mix easyb stories and specifications in with the usual Grails tests and run them using "grails easyb-test" command. Note, however, that at this point, to execute both your easyb and usual Grails tests, you have to run two commands, both "grails test-app" and "grails easyb-test": just running the normal "test-app" won't also execute the easyb tests, and running the easyb tests won't also execute the normal tests.

So let's say I'm working with an application that takes dry cleaning orders and I want to test the behavior of the service that creates orders when a customer drops of some suits to be dry cleaned. Assuming I have a class called DropOffService, I'll create a new file in the test/unit directory, calling it either DropOffService.story or DropOffServiceStory.groovy (either will work).

Now I can define the behavior I want in the story file. I won't go in-depth on the details of writing easyb stories since there's good documentation on the easyb site for that, but the short form is that I create scenarios with given conditions, some event that occurs, and a pre-defined outcome. To test our service's behavior, I might define the story like this:

scenario "customer drops off suits", {
    given "a drop-off service"
    and "a customer with suits"
    when "the customer drops off the suits"
    then "the service should generate a new order"
}

At this point, I haven't actually written any code that will exercise the service, but I can still run "grails easyb-test". When I do, the plugin looks for valid easyb stories and specifications, runs them, and generates reports in the test/reports directory. The easyb plugin generates both text and html reports just like the standard Grails tests; however, it also produces XML output, too, potentially useful for reporting in something like a continuous integration system. Unlike the standard Grails reporting, instead of getting a file for each test class, the easyb plugin generates a report for each *type* of test. So we end up with a reports listing that looks like this:

test/reports/html:
easyb-functional.html  easyb_img04.jpg         easyb_report.css
easyb_img01.gif        easyb_img05.jpg         easyb_spacer.gif
easyb_img02.jpg        easyb_img06.jpg         easyb-unit.html
easyb_img03.jpg        easyb-integration.html  prototype.js

test/reports/plain:
specifications-functional.txt   stories-functional.txt
specifications-integration.txt  stories-integration.txt
specifications-unit.txt         stories-unit.txt

test/reports/xml:
easyb-functional.xml  easyb-integration.xml  easyb-unit.xml

If we actually look at the reports, we'll see that the single scenario I've written has a result of "pending" since I haven't actually implemented any code to exercise the drop-off service. At the point where I actually *do* something in the story, the result will change depending on whether or not the scenario produces the outcome I'm expecting.

So there's the basics of using easyb with Grails. There's definitely more to explore beyond this simple example (testing controllers, using dependency injection, and such), and I'll do that in future postings.

Binding the Grails DataSource in JNDI for Integration Testing

One of the components of the Grails application we're working on requires a database connection to be looked up via JNDI. That's fine for running in an application container in production where we can configure JNDI data sources easily enough, but for our integration tests using the embedded HSQLDB, we needed a way to get the Grails-configured testing data source via JNDI.

While we could have set up a source in jetty-env.xml and placed that file in WEB-INF, from a number of postings on the 'net (see, for example, this thread), it seems like the integration tests don't actually run within the web application context Jetty provides. So, instead, we decided to register the data source programmatically as we needed it in our integration tests.

Fortunately, Grails provides access easily enough to the testing data source (configured in grails-app/conf/DataSource.groovy) merely by adding a "def dataSource" in our integration test class defintion:

class MyClassTests extends GroovyTestCase {
    def dataSource // auto-wired by Grails

    . . . setup, test methods, etc . . .
}

With the data source defined, we can now access it as we need it.

The next step is to create an InitialContext and bind the data source:

def ctx = new javax.naming.InitialContext()
ctx.bind("java:/testing-db", dataSource)

Now, with the data source bound in JNDI, we can retrieve it and use it where we need it:

def ctx = new javax.naming.InitialContext()
def conn = ctx.lookup("java:/testing-db")
def sql = new groovy.sql.Sql(conn)

// create a table to hold some test data
sql.execute("create table test_data(id identity, some_column varchar, col_ts timestamp default current_timestamp)")

Of course, if we didn't want to worry about accidentally messing up the Grails testing data source, we could always have set up our own HSQLDB data source and bound it:

import org.apache.commons.dbcp.BasicDataSource

def ds = new BasicDataSource()
ds.setUrl("jdbc:hsqldb:mem:myTestDb")
ds.setUsername("sa")
ds.setPassword("")
ds.setDriverClassName("org.hsqldb.jdbcDriver")

def ctx = new InitialContext()
ctx.bind("java:/testing-db", ds)

However, in our case, using the standard Grails data source worked just fine. It's probably worth our while to try to set up a data source through jetty-env.xml for using our components requiring JNDI in the "run-app" development environment, but our integration tests are running, and that's good enough for now.

Pretty-printing XML results returned from Groovy RESTClient

In another blog post here, someone commented that they had to fall back to HTTPBuilder to output the XML returned from a REST request because RESTClient automatically parses the XML and returns a GPathResult NodeChild object. That's certainly one solution (and it gives you a good amount of control over both the request and its output), but if you *really* want to use RESTClient, it's still possible to format the results parsed via XMLSlurper.

When I needed to do this, I ended up using a combination of Groovy's StreamingMarkupBuilder to create the initial output, then I used Xalan to transform that output into easier to read XML. The method ended up looking like this:

import groovy.xml.StreamingMarkupBuilder

import javax.xml.transform.TransformerFactory
import javax.xml.transform.Transformer
import javax.xml.transform.OutputKeys
import javax.xml.transform.stream.StreamResult
import javax.xml.transform.stream.StreamSource

/**
 *  pretty prints the GPathResult NodeChild
 */
def outputFormattedXml(node) {
    def xml = new StreamingMarkupBuilder().bind {
        mkp.declareNamespace("":node.namespaceURI())
        mkp.yield(node)
    }

    def factory = TransformerFactory.newInstance()
    def transformer = factory.newTransformer()
    transformer.setOutputProperty(OutputKeys.INDENT, 'yes')

    // figured this out by looking at Xalan's serializer.jar
    // org/apache/xml/serializer/output_xml.properties
    transformer.setOutputProperty("{http\u003a//xml.apache.org/xalan}indent-amount", "2")
    def result = new StreamResult(new StringWriter())
    transformer.transform(new StreamSource(new ByteArrayInputStream(xml.toString().bytes)), result)

    return result.writer.toString()
}

As I mentioned in the code comments, there were some pieces I had to figure out from looking in Xalan's serializer.jar. For modifying other output properties, take a look at org/apache/xml/serializer/output_xml.properties in serializer.jar to see just what you can customize and how to refer to it in your code.

In the end, the above method could probably be used to pretty-print any NodeChild object from GPathResult. There may be an easier or "more Groovy" way to output this kind of thing, but this allows the use of both the easy-to-use RESTClient and the easy-to-use XmlSlurper without having to worry much about making the output human-readable.

Grails type conversion outside of domain objects

In a recent Grails project, we needed a way to do type conversion outside of merely marshalling values for a domain object (i.e. outside of, say, automatically loading up the properties of a domain object from request parameters, a job which Grails does really well). Specifically, we needed to convert the String representation of a Date into an actual Date object for use in our domain object's findByLastUpdatedGreaterThan() method. Initially, we hoped the method would automatically convert the type for us, but instead it threw exceptions saying "No signature of method . . . is applicable for argument types" because it couldn't handle a String instead of a Date.

When Grails converts types for marshalling request parameters to domain objects, behind the scenes, it uses a GrailsDataBinder instance, converting values in the process of loading up the object's properties. And while the GrailsDataBinder implements Spring's TypeConverter and gives us access to the convertIfNecessary() method, it also requires a domain object instance in its constructor so it has a target to bind values to. Since all we were doing is simple type conversion, we wanted a different way.

Enter Spring's SimpleTypeConverter, a class that really only does one thing: convert from a String type to an instance of a target class. SimpleTypeConverter allows us to register custom property editors, the backbone of the type conversion process, so the first task was to implement a PropertyEditor (extending PropertyEditorSupport) that would let us take a series of date formats and parse them to a Date. Thanks to some very detailed information in a response to this StackOverflow Question, implementing our CustomDateEditor was fairly easy:

import java.beans.PropertyEditorSupport
import java.text.SimpleDateFormat
import java.text.ParseException

/**
 * A property editor that allows for handling many different
 * date formats at once (as opposed to Spring's default CustomDateEditor
 * which only handles one date format configured at instantiation)
 *
 * This class is modified from information found at:
 * http://stackoverflow.com/questions/963922/grails-date-unmarshalling
 */
class CustomDateEditor extends PropertyEditorSupport {

  private final List< String> formats

  CustomDateEditor() {
    // in our production code, we get the formats from our Grails config
    def formats = [ "yyyyMMdd-HHmmss", "yyyy-mm-dd HH:mm:ss.S" ]
    List< String> formatList = new ArrayList< String>(formats.size())
    formats.each { format ->
      formatList << format.toString() // Force String values (eg. for GStrings)
    }
    this.formats = Collections.unmodifiableList(formatList)
  }

  def String getAsText() {
    return this.value.toString()
  }

  def void setAsText(String s) throws IllegalArgumentException {
    if (s) {
      formats.each { format ->
        def df = new SimpleDateFormat(format)
        try {
          this.value = df.parse(s)
          return
        } catch (ParseException e) {
          // Ignore
        }
      }
    }
  }
}

Next, we wanted to make this custom Date editor available across the application, so we added it to our resources.groovy Spring config:

beans = {
    customDateEditor(CustomDateEditor) { bean ->
        bean.singleton = false // editors store state so are not thread-safe
    }
}

Now anywhere we wanted to do type conversion, all our classes needed to do was create a new SimpleTypeConverter, get a CustomDateEditor instance from Spring, and register that editor with the SimpleTypeConverter:

// grailsApplication auto-wired by Grails
def editor = grailsApplication.mainContext.getBean("customDateEditor") 
def converter = new SimpleTypeConverter()
converter.registerCustomEditor( Date.class, editor )

With the converter set up and our custom Date editor registered, it was now a simple matter of doing the conversion:

// NOTE: production code determines type dynamically from field being queried
def type = Date.class
def converted = converter.convertIfNecessary( value, type )

If we had wanted to make our custom editor available for marshalling values to a domain object, we could have easily created an PropertyEditorRegistrar object and configured it via Spring so Grails would automatically use our property editor (as described in the StackOverflow Question above). But, again, all we wanted was fairly straightforward type conversion outside of a domain object. The combination of Spring's SimpleTypeConverter and our CustomDateEditor gave us that.

Grails integration testing: GroovyTestCase v. GrailsUnitTestCase

Working on a Grails project in IntelliJ over the last few weeks, I've accumulated a number of unit and integration tests for various classes. Up until now, since I've only been working on isolated sections, I've been running the test classes individually, but I finally did a refactor that cut across a number of areas that really required me to run most of the tests. Since I was feeling a bit fuzzy headed with a late-summer cold, I started running the tests one by one, each one passing, but I quickly realized that was going to be a major pain.

Since Grails allows you to run all the tests at once, I figured I'd just run "grails test-app" and be done with it, but to my surprise, a number of tests which had passed when I ran them individually failed when I ran the tests in one batch. Looking more closely and doing a bit more logging, it turned out that my controller classes that did content negotiation and used a "withFormat" block to check for XML content weren't handling the XML content: they were either running the html/form section rather than the section for handling xml content or they were running nothing at all.

After a couple hours of head scratching and trying different approaches, it looked like the Grails configuration properties (which contain pieces telling Grails how to deal with Accept headers and different MIME types) weren't being made available to test cases in the integration tests. In a bit of desperation, I basically put the configuration parameters for doing content negotiation in one of my controller integration test case's setUp() methods:

def grailsApplication
void setUp() {
    super.setUp()
    grailsApplication.config.grails.mime.file.extensions = true
    grailsApplication.config.grails.mime.use.accept.header = true
    grailsApplication.config.grails.mime.types = [ html: ['text/html','application/xhtml+xml'],
                      xml: ['text/xml', 'application/xml'],
                      text: 'text/plain',
                      js: 'text/javascript',
                      rss: 'application/rss+xml',
                      atom: 'application/atom+xml',
                      css: 'text/css',
                      csv: 'text/csv',
                      all: '*/*',
                      json: ['application/json','text/json'],
                      form: 'application/x-www-form-urlencoded',
                      multipartForm: 'multipart/form-data'
                    ]
}

Magically, it worked. And not only that, it magically also made the other controller tests work, too, even without editing their setUp() methods. It seemed something was bleeding over and/or not getting cleaned up somewhere.

I mentioned all this chatting with my friend Eric (see his blog at Sword Systems for some really great insights on Java, Groovy, and tools of the trade) who proceeded to tell me that he discovered some weirdness with GrailsUnitTestCase and how it mucked with the environment. I looked at the test cases I had (mostly generated automatically by Grails/IntelliJ), and they were a mix of GrailsUnitTestCase (integration tests for domain classes) and GroovyTestCase (integration tests for controllers, the failing tests). It turns out that the GroovyTestCase tests would work just fine on their own, but when mixed in the same environment with the GrailsUnitTestCase, somehow they lost their access to the Grails config.

Doing a bit more research, I found this Grails bug report, which mentions that "GrailsUnitTestCase sets grailsApplication.config to null in its tearDown routine which causes any additional tests . . ." to have problems. It seems even with this known problem, Grails still generates integration tests extending from GrailsUnitTestCase.

So the solution was to use GrailsUnitTestCase (or its derivatives like ControllerUnitTestCase) *only* for unit tests (in spite of the examples at pages like Grails - Testing Controllers); for integration tests, use GroovyTestCase (like the examples in the main Grails documentation on testing). Once I changed all my integration tests to GroovyTestCase and ran them all as a batch, they all passed. I took that ugly configuration code out of the setUp() method and ran it again, and they all passed. Problem solved (thanks, Eric!).

Groovy RESTClient, HTTPBuilder, and PUT responses with no body

In working with a RESTful web service that allows PUT requests to upload files, I came across an slight annoyance with the put() convenience method in the Groovy RESTClient. Up until now, I've been absolutely amazed with how easy RESTClient makes my life in working with RESTful web services, and I'll continue to use it in doing REST development. However, in this one case, RESTClient wouldn't work nicely for me.

Here's the scenario: I upload a zip file to the service so that it can use the data in that file to populate some things (see my post Groovy, RESTClient, and PUTting zip files for one way to get RESTClient to handle zip files). I watch the log on the server and see that it's handling that file just fine; however, my Groovy script errors out! What's going on?

Here's what the Groovy code looked like at this point:

def file = new File("data_file.zip")
def rest = new RESTClient( 'http://localhost:8080/server/rest/' )
rest.encoder.'application/zip' = { ... handle zip files ... }
rest.put( path: "data/data_file.zip", body: file, requestContentType: 'application/zip' )

It turns out that the RESTful service, once it gets the file, just sends back an HTTP "201 Created" response code. No body to the response at all because, really, there's none needed. However, by default, the RESTClient put() method wants a response body to parse and return, so it throws a NullPointerException when it tries to parse that "nothing".

Since I didn't see a way of telling RESTClient not to expect a response body, I could have just lived with the exception, catching it and moving along. But the exception was being thrown before the response object was passed back, I couldn't even check the status of my request to see if the request was successful or if there were problems. Fortunately, RESTClient is really just a set of convenience wrappers around the main HTTPBuilder, so I was able to implement my request using HTTPBuilder.

HTTPBuilder has the nice feature of allowing you to assign different response handlers to different HTTP response codes, so I could basically tell it not to bother trying to parse '201' responses and just print a "success" message. Here's how I ended up setting up the PUT request using HTTPBuilder:

import groovyx.net.http.HTTPBuilder
import groovyx.net.http.Method

def file = new File("data_file.zip")

def http = new HTTPBuilder('http://localhost:8080/server/rest/')
http.encoder.'application/zip' = { ... handle zip file ... }

http.request(Method.PUT) { req ->
    uri.path = 'data/data_file.zip'
    send("application/zip", file)

    response.'201' = { resp ->
        println resp.status
    }

    response.success = { resp, object ->
        println resp.status
    }

    response.failure = { resp ->
        println resp.statusLine
    }
}

Of course, we could do a lot more with the response objects, but at this point, all we want to know is whether our request failed or succeeded, without having to worry about a response body. While it's definitely not as convenient as RESTClient, with the HTTPBuilder implementation in place, the file uploaded just fine, and there were no errors on the Groovy end of things.

Groovy RESTClient and PUTting zip files

I'm currently working with a RESTful web service that requires a client to upload a zip file with a content-type of 'application/zip'. Since we're using more and more Groovy in our shop, we initially tried to PUT the zip file using Groovy's HTTPBuilder/RESTClient. Our initial attempt looked like this:

def file = new File("data_file.zip")
def rest = new RESTClient( 'http://localhost:8080/server/rest/' )
rest.put( path: "data/data_file.zip", body: file, requestContentType: 'application/zip' )

When we first tried to run it, we kept getting NullPointerExceptions from RESTClient/HTTPBuilder trying to set the body of the request. Digging into the code, it looked like, by default, HTTPBuilder doesn't know how to handle zip files. It can do other kinds of binary encoding, but the content-type needs to be 'application/octet-stream', something the server we were using doesn't understand.

What we had to do was actually create our own encoding process and register that with the RESTClient. Using the HTTPBuilder EncoderRegistry.encodeStream() method (which returns InputStreamEntity instances of org.apache.http.HttpEntity) as a starting point, here's what we came up with:

/**
 * Request encoder for a zip file.
 * @param data a File object pointing to a Zip file on the file system
 * @return an {@link FileEntity} encapsulating this request data
 * @throws UnsupportedEncodingException
 */
def encodeZipFile( Object data ) throws UnsupportedEncodingException {
    if ( data instanceof File ) {
        def entity = new org.apache.http.entity.FileEntity( (File) data, "application/zip" );
        entity.setContentType( "application/zip" );
        return entity
    } else {
        throw new IllegalArgumentException( 
            "Don't know how to encode ${data.class.name} as a zip file" );
    }
}

Basically, what we're doing is, instead of returning an InputStreamEntity, we return a FileEntity and set it's content-type to 'application/zip'. Of course, in the above code, we could do more strenuous checking on the data to make sure that it's actually a zip file and such. As well, we could probably expand it to handle other specific binary types. But in this case, we knew we were getting a zip file and nothing else, so YAGNI.

Once we had that method in place, all we needed to do was register it with HTTPBuilder/RESTClient. HTTPBuilder/RESTClient allows access to its encoders Map, and its propertyMissing() setter implementation automatically registers an encoder to that Map, so it was easy to attach our zip file encoder to our client object:

rest.encoder.'application/zip' = this.&encodeZipFile

With that in place, uploading the zip file to the RESTful web service worked. Our final code looked something like this:

def file = new File("data_file.zip")

def rest = new RESTClient( 'http://localhost:8080/server/rest/' )
rest.encoder.'application/zip' = this.&encodeZipFile
rest.put( path: "data/data_file.zip", body: file, requestContentType: 'application/zip' )

def encodeZipFile( Object data ) throws UnsupportedEncodingException {
    if ( data instanceof File ) {
        def entity = new FileEntity( (File) data, "application/zip" );
        entity.setContentType( "application/zip" );
        return entity
    } else {
        throw new IllegalArgumentException( 
            "Don't know how to encode ${data.class.name} as a zip file" );
    }
}

Grails, JSecurity, and View URL Mapping in JBoss

I've been trying to get my app created in Grails 1.1.1 using the JSecurity plugin to deploy in JBoss 4.2.3.GA. Everything worked fine when I test it using 'grails run-app', but when I deployed my WAR file to JBoss, I got an IllegalStateException any time I tried to access something where I mapped a URL to a view in my UrlMappings.groovy file like this:

"/"(view:"index")
"500"(view:"error")

It turns out that there's a bug open in the JSecurity plugin: GRAILSPLUGINS-1117. Basically, there's not much explanation as to why, but there is a work-around. Instead of mapping to a view in the UrlMappings file, you map to a controller and action. In the case of the error view, it would look something like this:

"500"(controller:"errors", action:"serverError")

However, there isn't a nice way (that I've found, at least) to map the default "index" view. Instead, you'll either have to create a custom "home" controller that forwards to your generic index page, or map the "/" URL to one of your existing controller actions. In the case of my application, since the main entry point will eventually be through one of the existing controllers anyway, I ended up mapping "/" to that:

"/"(controller:"entryController", action: "list")

With those changes, the app works just fine under JBoss 4.2.3.GA. The JSecurity plugin bug is currently marked as major, so I hope at least it's getting some attention and will be fixed in the near future.

Enabling the "Accept" header for Grails/REST

I've been trying to implement a RESTful web service using Grails, following along a number of articles on the web for examples (Scott Davis has a particularly good one here: RESTful Grails). To get back XML from a Grails app that will be serving up both HTML and XML, I was setting the HTTP "Accept" header, then using content negotiation in my controller classes to determine what to send back to the client. For some reason, though, my app kept spitting back HTML even when I was specifically requesting XML.

Well, it turns out that, according to this Jira post, as of Grails 1.1, using the Accept header is disabled by default. To enable it, open up the Grails app's Config.groovy file (grails-app/conf/Config.groovy), and set "grails.mime.use.accept.header" to true.

Once I got that figured out, everything is running really smoothly.

Playing a nicer beep in Gnome terminal under Ubuntu

With all the tab-completion and such that I do at the command line, sometimes that system beep can get really annoying. I like to have audio feedback when I'm working, so I didn't want to disable the beep completely, but I did want something that wasn't quite so harsh. I searched around for a while in the preferences to see if I could change things to play a nicer sound file instead of the speaker's beep, but it doesn't look like that's available as an option. Searching the web, I found this post at tjworld.net, which got my system playing the sound file I wanted: XandGnomeSystemBeepWithAudioFile. (Note: there's a misspelling for the xkbevd config file name in that post; it's actually "xkbevd.cf".)

While that finally got a sound file playing instead of the beep in other apps, in the Gnome terminal, it played both the sound file *and* the system beep . . . even more annoying :). In the end, here's what I had to do to get *only* the sound file playing as the beep in the Terminal:

First, in the System->Preferences->Sound preference setter, go to the System Beep tab, and de-select "Enable System Beep".

Next, in the Gnome terminal Edit->Profiles menu, select the profile you're using, then on the Edit->General tab make sure "Terminal bell" is selected.

So, you have to *disable* the system beep in the global settings, then *enable* it in the terminal settings.

Accessing the Maven runtime classpath in a Groovy script

Lately, I've been involved in converting a good portion of our source tree that's currently built with Ant over to building with Maven. We have a number of Groovy scripts that we use throughout our build, and I've had great success with the GMaven plugin to get things working. I ran into one snag, though, when I was trying to run one of our Groovy scripts that generates Textile wiki markup via a custom Javadoc doclet.

To get the doclet to run, I needed to supply the path/classpath where it could be found, and since the doclet is built during the project's compile phase, I needed access to the Maven runtime classpath to get the documentation built (admittedly, it shouldn't be part of the "project", but if it were, instead, an outside dependency, I'd still need access to the Maven compile classpath, so the problem is pretty much the same). The Maven AntRun plugin provides a nice reference to that classpath via the "maven.runtime.classpath" property that can be accessed in the Maven Ant targets, so I was hoping that GMaven would provide the same sort of thing, especially since they give good access to an AntBuilder automatically.

At this point, though, they don't, and I really didn't want to resort to embedding Ant in our Maven build just to run a Groovy script (for a number of reasons). After a good amount of experimentation and searching, I found this GMaven Jira entry that gives a little bit of guidance for how a GMaven Mojo might build those same kind of classpath references. It didn't really address how to get those classpaths in an executed script, though, but Groovy is Groovy, so I figured it could be done :). Experimenting with what's laid out there and digging into the Ant API docs, here's a what I came up with:

First, of course, you need to get GMaven to run during your build. The GMaven documentation is great for showing how to edit your POM to call a Groovy script during the build. Next, in the Groovy script you want to run, you need to import org.apache.tools.ant.types.Path, and instantiate a new Path object (before you want to reference the path, naturally). When you make a new Path object, you need to give it a reference to the Ant Project to which it belongs, and, as I mentioned, GMaven gives you reference to an AntBuilder (via the "ant" default variable) where you can get that project. The code so far might look like this:

import org.apache.tools.ant.types.Path

Path runtimePath = new Path(ant.antProject)

Once you have that Path object, you need to tell it what its path should look like. Since we're trying to set the runtime classpath Maven builds, we need to find out what JAR files and compiled classes are in Maven's runtime classpath. Fortunately, GMaven also gives us access to a MavenProject object by default, referenced by the default variable "project" in the script. We can build up our path, then, by joining the different pieces of the MavenProject's runtimeClasspathElements and setting that to our Path's path:

runtimePath.path = project.runtimeClasspathElements.join(File.pathSeparator)

Finally, in order to reference that path by name in our script where we need it, we have to add a reference to it in our Ant Project object:

ant.antProject.addReference('runtime.classpath',runtimePath)

The final code for building the runtime classpath looks like this:

import org.apache.tools.ant.types.Path

Path runtimePath = new Path(ant.antProject)
runtimePath.path = project.runtimeClasspathElements.join(File.pathSeparator)
ant.antProject.addReference('runtime.classpath',runtimePath)

Once we have that defined, we can reference it in the script just like we would any other Ant reference. For example, in the project I'm working on, I need to supply a path reference to the Ant Javadoc task. Since I named the path above "runtime.classpath", I set up my Javadoc task call in the Groovy script like this:

ant.javadoc(... docletpathref: 'runtime.classpath' ...) {
  ... more Groovy code to execute ...
}

While it would definitely be easier if GMaven provided these classpaths out of the box, it's actually not that hard to build them up in the script when you need them. By the way, you can do the same thing with the Maven compile classpath: instead of referencing "project.runtimeClasspathElements" when building the path, you use "project.compileClasspathElements". If you need the plugin classpath, the process is a little more complex, but if you get what's going on above, then you can probably follow the example in the GMaven Jira post to build the plugin classpath.