CI with Hudson, CF, and MXUnit: Configuring ANT and Running Tests

Friday, May 29, 2009

In the last entry, I showed how to get Hudson set up and connected to your SVN server. In this entry, I'll demonstrate how to run your MXUnit tests in a Hudson project.

If you’re already running your MXUnit tests with ANT, then this should go quickly. If not, then you’ll need to read this first. Create your ANT build file (build.xml), create your “runTests” target (or whatever you want to call it), and get your test runs working from within Eclipse or at the command line. Only after you get those working should you try integrating with Hudson… one thing at a time.

Configuring ANT within Hudson

Hudson needs to know where your ANT install lives. If you haven’t downloaded ANT, do so now, unzip it, and put it somewhere. Then, copy the path to that location.

From your main Hudson screen, click on “Manage Hudson”, then “Configure System”. More quickly, type “configure” into the search box and it’ll take you to the same place. Go down to the “ANT” section, click “Add”, and paste that path to your ANT install. Note, this is NOT the “bin” directory… just the ANT root. Give it a name, like “ANT 1.7” or whatever. Click Save

Hudson will not treat this ANT location as the default location. You must (at least, I did) have to explicitly choose this ANT instance for every project setup. Just so you know. This tripped me up at first.

Sample ANT build file

<?xml version="1.0" encoding="ISO-8859-1"?>

<project name="Sample" basedir="." default="main">

<target name="init">

<property name="testroot" location="../tests/framework" />
<property name="componentPath" value="mxunit.tests.framework" />
<property name="server" value="localhost" />
<property name="port" value="80" />

<!-- wherever the mxunit-ant.jar lives -->
<property name="mxunitant" location="../ant" />
<property name="mxunitjar" location="${mxunitant}/lib/mxunit-ant.jar" />
<property name="mxunitxsl" location="${mxunitant}/xsl" />

<!-- this is where the xml and html will live -->
<mkdir dir="testresults" />

<target name="runtests" description="Make output directories and run the MXUnit task" depends="init">
<taskdef name="mxunittask" classname="org.mxunit.ant.MXUnitAntTask" classpath="${mxunitjar}" />
<mxunittask server="${server}" port="${port}" outputdir="testresults" verbose="true">
 <directory path="${testroot}" recurse="false" packageName="${componentPath}" componentPath="${componentPath}" />

<mkdir dir="junithtml" />
<junitreport todir="junithtml">
 <fileset dir="testresults">
  <include name="*.xml" />
 <report format="frames" todir="junithtml" styledir="${mxunitxsl}" />


Configuring your project’s test runner step

Navigate to your project, then click “Configure”. More quickly, you can search for your project by name, then search for “configure”, and you’ll get to the same screen.

Find the "Build" section. click "Add build step". Select "Invoke ANT". Select the ANT instance from the dropdown list of installed ANTs. In the "target" field, type the name of the target in your ANT build file that you use to run tests. For example, "runTests"

This might be all you need. so you could save this and try to do a build now, and see what happens.

Test Runner Step Configuration, Part 2: Properties

Now’s a good time to talk about the weirdo nature of ColdFusion projects with respect to CI. Remember, you checked out your project from Subversion into Hudson’s workspace, but that’s not the code you care about… your code already lives on a DEV server and is updated by some other means. However, there is one file that gets checked out that you do care about: Your ANT build file.

When Hudson runs your ANT file, it’ll be running the one from its workspace. And then that build file will be executing its tests against the components you specify on your DEV server. Got it? Good. Confused? I know.

It’s possible that your build file’s runTests target might need some properties that are possibly set in a properties file that you don’t keep in SVN or which otherwise you need to override. For example, for MXUnit itself, our build file looks at a file named “” to get some configuration information such as server name, port, usernames/passwords, webroot location, and some other things. This is NOT kept in SVN b/c it’s unique for each MXUnit contributor. When MXUnit’s runTests is run, it tries to read that properties file and then set the server, port, and webroot properties. But since that file isn’t checked out from SVN, it’s not visible in Hudson’s workspace and thus those properties never get set. It’s super easy to get around this though!

In your ANT step’s “Advanced” tab, you’ll see a “properties” box. In here, you can type in any properties in name=value format, linebreak-delimited, and poof, properties problem solved. For example, here’s what I put into the properties box for the MXUnit job:

server=localhost port=80 webroot=c:/inetpub/wwwroot

Configuring your project’s JUnit XML Output

In that sample ANT build file above, you saw how easy it was for MXUnit to spit out JUnit-compliant XML. Basically, no matter where your tests actually run (http://blah/whatever/HttpAntRunner.cfc), your JUnit XML files will go in a directory relative to the build file that is running. Got that? That means that, as discussed above, your JUnit xml files will go in your Hudson workspace somewhere. This is perfect, because that’s where Hudson wants them to be.

For example, with the MXUnit build, the build file is at mxunit/build.xml. It specifies a location for the JUnit xml output as something that evaluates to tests/testresults/tmp/. So in the Hudson configuration for the MXUnit job, I specify the JUnit XML location as:


That’s it. So wherever your xml files go relative to your workspace root, simply go to the “Post Build Action” section of your job configuration, check the “Publish JUnit XML” checkbox, and enter in the path to your xml files, with *.xml, as I did above. Click save

Re-saving ant.bat

After I got the MXUnit project all set up, I ran the build; it executed the tests; Hudson read the JUnit XML (all tests passed). And then…. Failed. Failed?! WTF? WTF was failing? The Hudson console output (thank you, Hudson!) read: “The system cannot find the batch label specified”.

This wasn’t a Hudson problem. It wasn’t an MXUnit problem. Turns out, it was the ant.bat file giving me grief. That’s right… the ant.bat that ships with the eclipse ANT install. Perhaps had I been using a “proper” install, this wouldn’t have been the case. Nonetheless, I wasn’t. So, how to fix it? Simple

Open up ant.bat in Textpad. Notice that it’s in Unix format. Change the format to “PC”. Click Save.

Rerun the Hudson build. Oh Joy. Oh Rapture. A fully functioning test run with test results. Let the CI begin.

What’s Next?

Next up, configuring notifications.


Schalk Neethling said...

Thanks so much! You saved me a heck of a lot of pain.

OpenSource Release Feed

Raymond Camden said...

I've run into a bug that is either an issue with mxunit or junit. I'm not sure which. I was working on unit tests and using Hudson to run them. (Yeah, kinda an indirect way to just run UTs, but whatever.) In one on my services, I set a cookie in the response and the unit tests run this particular service. The cookie contains JSON info. At some point, mxunit writes out a file called:


In that XML there are numerous property tags. They seem to be all the CGI vars and my cookie was in there as well.

However, it is not properly escaped, so I end up with this in the xml:

As you can see, that is obviously not good XML. This then makes junit throw an error that I can see via hudson:

[junitreport] The file c:\Users\Raymond\.hudson\jobs\NewHotness\workspace\newHotness\maverick\unittests\testresults\mxunitdirectorytestsuite_1.xml is not a valid XML document. It is possibly corrupted.

This then causes the build to fail. Any ideas?

Raymond Camden said...

Ugh, damn blogger escaped the xml I posted:

property name="AUTOAUTH" value="{"P":"(9\\B +\\RCJ5< \n","UID":"C27BBF3F-E309-36AC-51D3A9E6275A9EB7"}" /

I remove the closing and opening tags above.

Marc Esher said...

I'll take a look at this, Ray

Steven Erat said...
This comment has been removed by the author.
Steven Erat said...

Ray - Better (18 months) late than never, but I got the "mxunitdirectorytestsuite_1.xml is not a valid XML" too and found that it was due to the application using CFHTMLHEAD to write some layout code to the output stream. This was stomping on the mxunit xml. There was some logic around the CFHTMLHEAD tag to exclude it in some cases, so I modified it to not run that line if the cgi.script_name contains HttpAntRunner.cfc, and that fixed it.