Adventures in Mocking, Part 1

Wednesday, April 23, 2008

For the longest time, I never used mocks in unit tests. I guess I never truly understood them. Today, I finally "got" it. Or, at least, I now get one slice of the pie. Here's a function I want to test. It's named rollBackFastPlanFromProduction:
<cffunction name="rollBackFastPlanFromProduction" output="false" access="public" hint="rolls back a fastplan from production if no orders are pending. Returns a struct with keys success and message." returntype="struct">
<cfargument name="planid" required="true" type="numeric">
<cfargument name="plandocid" required="true" type="numeric">
<cfargument name="languageid" required="true" type="numeric">

<cfset var prevmsg = "">
<cfset var msg = "">
<cfset var s_result = StructNew()>

<!--- Before doing anything make sure pending orders won't hold this request up... --->
<cfset var OutstandingOrders = selOutstandingOrdersList(arguments.planid,arguments.plandocid,arguments.languageid)>

<cfif len(trim(OutstandingOrders)) EQ 0>
   <!--- If no outstanding orders come back, proceed... --->
   <cfset prevmsg = UpdatePlanStatus(PlanID=arguments.PlanID,PlanDocID=arguments.PlanDocID,PlanStatusID=2,checkPlanStatusChange=0)>
   <cfset msg = "Plan rolled back from Production. " & ListLast(prevmsg,";")>
   <cfset s_result.success = true>
   <cfset s_result.success = false>
   <cfif ListLen(OutstandingOrders) GT 1>
       <cfset msg = "Rollback unsuccessful! Orders with confirmation numbers: #OutstandingOrders# still unfulfilled!">
       <cfset msg = "Rollback unsuccessful! Order with confirmation number: #OutstandingOrders# still unfulfilled!">

<cfset s_result.message = msg>
<cfreturn s_result>
This is pretty simple code: basically, see if there are any outstanding orders. If Not, run the updatePlanStatus function; otherwise, just set a message and return the result struct. (NOTE: this method of returning structs is not best practice but it follows a model we have at work and thus I'm choosing consistency in this case. ) Ok, so, I want to test this code. All I want to know is:
  1. When there is a single outstanding order, does the update function NOT run, and does the struct return a false "success" key
  2. When there are multiple outstanding orders, does that same behavior apply
  3. When there are no outstanding orders, does the update function run and return a true "success" key
That's pretty much it. I'm not interested in the particulars of the message. And I surely don't care about the particulars of the results of UpdatePlanStatus... that code may or may not be tested elsewhere but it's not this test function's job to test that. Right away, here are the problems:
  1. how to create the conditions for a single outstanding order
  2. how to create the conditions for multiple outstanding orders
  3. how to get the UpdatePlanStatus function to NOT do anything (it updates the DB, and I don't want to touch the DB in this test)
  4. how to get all of this to work without needing real data
This, in fact, is one of the core problems you seem to always run into in unit testing: how to set up your conditions without jumping through a million hoops. Back in the day, I'd have done this:
  1. Go into the database and find an ID in the appropriate table that would give me a single outstanding order
  2. Do the same for multiple outstanding orders
  3. Find an ID for no outstanding orders
Don't worry about the details: the point is that I'd have found data that matched the criteria I was trying to achieve and then, in the unit test, I would have literally used those IDs as arguments to my functions. This would NOT have solved my problem of not wanting to actually run any DB updates. It'd have just gotten the logic to work out. Well... it'd have gotten it to work out until the data in the DB changed. Then, all I'd have is a brittle test. The thing is that you want your tests to work all the time, even a month after you wrote them. Creating tests that depend on certain conditions in the DB itself are a sure way to brittle tests. I know because I've written dozens upon dozens of these kinds of tests. Now then, onto Mocks. For this post, I'm not going to use a mock framework. Just good old fashioned ColdFusion. Here are my goals:
  1. replace the UpdatePlanStatus function inside my object under test with a new version of the function that simply returns the string "updated".
  2. replace the selOutstandingOrdersList function with 3 different functions, depending on what i'm trying to test. In one test, it'll return an empty string. In another test, it'll return a single number. And finally, it will return a comma-delimited list of numbers.
First, my setUp function:
<cffunction name="setUp" returntype="void" access="public" hint="put things here that you want to run before each test">
<cfset pdd = createObject("component","")>
<cfset pdd.init(dsn=application.maindsn)>
<!--- so we can easily mock! --->
<cfset pdd.mixin = mixin>
So we have an object we'll name "pdd". But the fun part here is this: set pdd.mixin = mixin WTF?
    <cffunction name="mixin" access="private">
  <cfargument name="method" type="string" required="true">
  <cfargument name="value" type="any" required="true"/>
   <cfset this[method] = value>
   <cfset variables[method] = value>

There, that's better. I created a new, private function inside my test named "mixin". So, up in setUp, I'm attaching my new function to the pdd object. What mixin does is a tiny bit of CF magic. Mad props to Nathan Strutz, because without this post I'd still be banging my head against the wall. More on that later. To see what it does, let's look at the next chunk of code. A new, mock function:
    <cffunction name="updatePlanStatus" access="private">
   <cfreturn "updated">
It's private so that MXUnit won't run it. All it does is return the string "updated". This means nothing until you look way back up top and see that the function under test makes a call to updatePlanStatus. And that's one of the functions I want to overwrite. So... how to get my new function into my pdd object? Mixin!
    <cffunction name="PlanRollBackShouldFailWithSingleOutstandingOrder" returntype="void" hint="">
    <!--- mock em so we don't touch the DB --->    
    <cfset pdd.mixin("updatePlanStatus",updatePlanStatus)>
    <cfset pdd.mixin("selOutstandingOrdersList",selOutstandingOrdersListOneResult)>
    <cfset s_result = pdd.rollBackFastPlanFromProduction(1,1,1)>
    <cfset debug(s_result.message)>
    <cfset assertFalse(s_result.success,"msg was #s_result.message#")>
    <cfset assertTrue(NOT findNoCase("updated",s_result.message),"#s_result.message# should not contain updated")>
See the call to pdd.mixin("updatePlanStatus",updatePlanStatus)? That's the magic. I'm overwriting the original function with my new braindead mock. This way, I don't touch the DB and I know if that function was run because if it is, it'll return the string "updated". But let's not worry about that right now, because it's honestly not important. The important part is that I've now got a really simple way to do mocking in CF when I need it. Note the next line in the function:
You should now be wondering, "What's selOutstandingOrdersListOneResult"? Here it is, a new private function added to my test case:
    <cffunction name="selOutstandingOrdersListOneResult" access="private">
     <cfreturn 1111>
So, as you see, it just returns a number, 1111 in this case. Thus, in the test above, what I want to ensure is that when the rollbackFastPlanFromProduction function is called, and it calls selOutstandingOrdersList, that that function returns a single-element list. Guess what I do to test multiple orders? If you're thinking "Create a new function that returns a multi-element list, and mixin that function instead... you're catching on nicely.
    <cffunction name="selOutstandingOrdersListTwoResults" access="private">
      <cfreturn "1111,2222">
And then, my new test:
    <cffunction name="PlanRollBackShouldFailWithMultipleOutstandingOrders" returntype="void" hint="">
       <!--- mock em so we don't touch the DB --->       
       <cfset pdd.mixin("updatePlanStatus",updatePlanStatus)>
       <cfset pdd.mixin("selOutstandingOrdersList",selOutstandingOrdersListTwoResults)>
       <cfset s_result = pdd.rollBackFastPlanFromProduction(1,1,1)>
       <cfset debug(s_result.message)>
       <cfset assertFalse(s_result.success,"msg was #s_result.message#")>
Now, I said I'd get back to why I'm thanking Nathan Strutz. I started this test with really simple code I thought would work: set pdd.updatePlanStatus = updatePlanStatus i.e. I had my new, private updatePlanStatus function and I wanted to overwrite the existing version in the object. Pretty simple, right? and when I called pdd.updatePlanStatus() directly, it did indeed return the string "updated". However, when the rollbackFastPlanFromProduction() function called updatePlanStatus, it ran the original code. Normally, I'm pretty good at figgerin' out this stuff. But after a half hour or so, I went all last-resort: I googled it. I don't know how, but I ran into Nathan's post, and he discusses what I honestly didn't know: that when functions call each other internally, they're calling the version of the function that lives in the variables scope. And when you, dear programmer, call a function on an object directly, you're calling it in this scope. So what I did not understand was that functions had two lives, so to speak: a life in variables scope, and a life in this scope. When I was overwriting the function using the direct cfset, I was overwriting it in the external -- this -- scope. But not the internal one. This brings me to the mixin function: the key line is this one:
<cfset variables[method] = value>
This little beaut right here overwrites the internal life of the function, and that's how I can create the mocks. So, thanks Nathan! Now, I leave you with some thoughts:
  1. this mixin business seems pretty handy, and I'd probably want to do it in all kinds of objects. Am I going to have to copy that function into every testcase?
  2. And wouldn't it be nice to NOT have to create those simple little private functions?
  3. And wouldn't it be nice to say "Hey, object, in this case, when you had a single outstanding order, did you NOT run that updatePlanStatus function? And in that other case, where you had no outstanding orders, did you run it?
This is where a framework for creating mock objects becomes valuable. And that's where I'll pick up next. Until next time... happy testing.


bill said...

dude, cohiba++. anxious to see what you come up with cfeasymock and coldmock. very interested, too, to see how and if you find benefit from behavioral verification (vs. state-based verification), and how to skin that cat ... not sure where _that_ phrase came from, but, the sound seems very appropriate when trying to refactor code to make it more testable.


Bob Silverberg said...

I just found this post via google as I'm trying to understand unit testing and mocking. Excellent post, really makes everything crystal clear. Now I'll have to read the rest of them.


Marc Esher said...

cool, glad it helped, Bob! Bill's been doing a lot with cfeasymock and i hope he blogs more about it soon. Also, the whole "hey, this mixin thing might be handy" thing in this blog post made its way into a tiny bit of new functionality in the 1.0 release of mxunit... we've not got a simple "injectMethod" function that all test cases have which lets you easily do this quick and dirty mixin mock stuff... for those who aren't ready for or don't yet need a mock framework.

Stay tuned... 1.0 will be out any day now.

Nathan Strutz said...

Oh hey, I didn't see this post until now. You posted it the week my daughter was born (4+ months back), so I was kind of out of the blogosphere for a while.

I'm glad to see my stuff helped someone. Of course, thanks right back to you for all the blogging and mxunit work you do.

Billigflug said...

Helped a lot! That´s great, once I got it, cause skillfully elaborated things aren´t always easy to understand:) Thanks anyway!!