Data-Driven testing with MXUnit's DataProviders

Wednesday, November 11, 2009

What are DataProviders?

Often in unit testing, you will want to run a test function for multiple inputs. Imagine you have a reformatFullName() that parses a string like “Smith, Bob” into “Bob Smith”. And in your test case, you would like to test your reformatFullName() function for a bunch of different inputs to ensure that you’ve covered common cases and also edge cases.

One common approach to this method is to specify the list/array/whatever – the collection of inputs – in the test function, and then write a loop inside the test function. This is a perfectly reasonable approach, though it does lead to some boilerplate.

DataProviders essentially remove the boilerplate; in other words, DataProviders offer a way for you to specify multiple inputs, but without having to construct the looping. In addition, in MXUnit, you have multiple built-in DataProviders at your disposal, which will be covered in other documentation and blog entries. For now, let’s look at a simple case: an array dataprovider.

How do I use DataProviders in my unit tests?

I Palindrome I

For this example, I wanted to have a little fun, so I chose the classic CS101 problem of Palindromes. I think back to my first programming course – Introduction to Programming in C – and distinctly remember the teacher’s frustrated attempts to drill recursion into our thick skulls. One of the problems he used to demonstrate it was Palindromes, i.e. How do you know if a word is the same forwards and backwards (“mom”, “dad”, etc)?  The teacher used rubber bands; he had us “act out” a palindrome. Oi, poor guy. Anyways…

Before we get into DataProviders, let’s start with a very simple test:

component extends="mxunit.framework.TestCase"{

	function setUp(){
		checker = new PalindromeChecker();		
	}

	function palindromeChecker_Should_ReturnTrue_ForPalindrome(){
		assertTrue(checker.isPalindrome("mom"));
	}

}

That’s a good start, but it doesn’t cover cases where the input isn’t a palindrome, and it sure feels incomplete, doesn’t it? Now, let’s look at a testcase that uses an array DataProvider. In here, we’ll specify an array of palindromes and tell MXUnit: Hey, MXUnit, use this array to run my test, one time for each element in the array:

component extends="mxunit.framework.TestCase"{

	function setUp(){
		checker = new PalindromeChecker();
		//set up dataprovider
		variables.palindromes = ["wow","mom","dad","eye","marccram","bob","poooooooooooop"];		
	}

	/**
	*@mxunit:dataprovider palindromes
	*/
	function palindromeChecker_Should_ReturnTrue_ForPalindromes(String theWord){
		//debug(theWord);
		assertTrue( checker.isPalindrome(theWord), "word #theWord# should have been flagged as a palindrome" );
	}

}

Here, we create an array named “palindromes”. Then, in the test, we specify the custom attribute “@mxunit:dataprovider” and set its value to “palindromes”. MXUnit will see that its been given a dataprovider, see that palindromes is an array, and will run the test one time for each element in the array. One other thing about this test function  should jump out at you: it takes a parameter. Typically, tests return void and accept no params. So why do we need this? Simply, MXUnit will pass the current value in the array into the argument which we’ve specified named “theWord”. If you run this test and uncomment the debug(theWord) statement, then hit ctrl-B on the test case in the MXUnit view in Eclipse, you’ll see each value as the test runs.

sshot-3

And now, let's add a negative test:

component extends="mxunit.framework.TestCase"{

	function setUp(){
		checker = new PalindromeChecker();
		//set up dataproviders
		variables.palindromes = ["wow","mom","dad","eye","marccram","bob","poooooooooooop"];
		variables.notPalindromes = ["woww","momm","marc","eyee","bbobbb","poooeeiop"];
	}
	

	/**
	*@mxunit:dataprovider notpalindromes
	*/
	function palindromeChecker_Should_ReturnFalse_ForNonPalindromes(String theWord){
		assertFalse( checker.isPalindrome(theWord), "word #theWord# should not have been flagged as a palindrome" );
	}

}

Now that we’ve got some tests – and they should be failing b/c we haven’t created a PalindromeChecker.cfc yet – let’s implement the functionality. We’ll do the braindead simplest way first (which is probably the way you would want to do it in real life):

/**
*@hint checks an input string for Palindrome-ness.
*
*/
component{	
	//how I'd program it if this were for a production system
	public boolean function isPalindrome_simple(String theWord){
		return theWord == reverse(theWord);
	}

}

And now let’s have some recursion fun:

/**
*@hint checks an input string for Palindrome-ness.
*
*/
component{
	//use recursion to get it; this is simply to demonstrate recursion
	public boolean function isPalindrome(String theWord){
		//the "base" case... the simplest case we can have that will return true
		if(len(theWord) <= 1) return true;

		/*compare the far left char with the far right char, and then pass the stuff in between
		the far left and far right back into isPalindrome
		 for input: marccram, calls will look thusly:
		   m == m && isPalindrome(arccra) is true
		   	a == a && isPalindrome(rccr)  is true
			 r == r && isPalindrome(cc)   is true
			  c == c && isPalindrome('')   is true
			   len('') <= 1 return true
		*/
		return left(theWord,1) == right(theWord,1) && isPalindrome( mid(theWord,2,len(theWord)-2) );
	}

	//how I'd program it if this were for a production system
	public boolean function isPalindrome_simple(String theWord){
		return theWord eq reverse(theWord);
	}

}

If you run your tests now, you should see them pass. For fun, you could put calls to isPalindrome() and isPalindromeSimple() in each of these tests as a sanity check.

How do I get these DataProviders?

Great question. As of this writing, the DataProvider functionality is in Subversion, and we’re prepping a release. If you just gotta have it right now, but you don’t want to check it out from SVN, let me know and I’ll whip up a nightly build.

Stay tuned for more documentation about the other new DataProviders in MXUnit!

--Marc

4 comments:

John Whish said...

Nice! I think this is a great addition :)

Can you can write cftag functions (for those of us still on CF8) and add a "mxunit:dataprovider" attribute?

Marc Esher said...

John, yes, you can use regular old attributes. Here's a quick example:


<cfset a = [1,2,3,4]>

<cffunction name="dataproviderShouldAcceptArrays" mxunit:dataprovider="a">
<cfargument name="arrayItem" />
<cfdump var="#arrayItem#">
<cfset assert(arrayItem gt 0 ) >
<cfset assertEquals(arrayItem,arguments.index ) >
</cffunction>

Bill's whipping up the docs and we should have this stuff out within a week.

Jamie Krug said...

This rocks. Test code, fast and clean :) Thanks!

Marc Esher said...

awesome, glad you like it, John. As you work with it, please let us know what needs to improve. I suspect we'll find bugs and also areas for making testing suck even less as we hammer more on this stuff