Adventures in Mocking, Part 1

Wednesday, April 23, 2008

For the longest time, I never used mocks in unit tests. I guess I never truly understood them. Today, I finally "got" it. Or, at least, I now get one slice of the pie. Here's a function I want to test. It's named rollBackFastPlanFromProduction:
<cffunction name="rollBackFastPlanFromProduction" output="false" access="public" hint="rolls back a fastplan from production if no orders are pending. Returns a struct with keys success and message." returntype="struct">
<cfargument name="planid" required="true" type="numeric">
<cfargument name="plandocid" required="true" type="numeric">
<cfargument name="languageid" required="true" type="numeric">

<cfset var prevmsg = "">
<cfset var msg = "">
<cfset var s_result = StructNew()>

<!--- Before doing anything make sure pending orders won't hold this request up... --->
<cfset var OutstandingOrders = selOutstandingOrdersList(arguments.planid,arguments.plandocid,arguments.languageid)>

<cfif len(trim(OutstandingOrders)) EQ 0>
   <!--- If no outstanding orders come back, proceed... --->
   <cfset prevmsg = UpdatePlanStatus(PlanID=arguments.PlanID,PlanDocID=arguments.PlanDocID,PlanStatusID=2,checkPlanStatusChange=0)>
   <cfset msg = "Plan rolled back from Production. " & ListLast(prevmsg,";")>
   <cfset s_result.success = true>
   <cfset s_result.success = false>
   <cfif ListLen(OutstandingOrders) GT 1>
       <cfset msg = "Rollback unsuccessful! Orders with confirmation numbers: #OutstandingOrders# still unfulfilled!">
       <cfset msg = "Rollback unsuccessful! Order with confirmation number: #OutstandingOrders# still unfulfilled!">

<cfset s_result.message = msg>
<cfreturn s_result>
This is pretty simple code: basically, see if there are any outstanding orders. If Not, run the updatePlanStatus function; otherwise, just set a message and return the result struct. (NOTE: this method of returning structs is not best practice but it follows a model we have at work and thus I'm choosing consistency in this case. ) Ok, so, I want to test this code. All I want to know is:
  1. When there is a single outstanding order, does the update function NOT run, and does the struct return a false "success" key
  2. When there are multiple outstanding orders, does that same behavior apply
  3. When there are no outstanding orders, does the update function run and return a true "success" key
That's pretty much it. I'm not interested in the particulars of the message. And I surely don't care about the particulars of the results of UpdatePlanStatus... that code may or may not be tested elsewhere but it's not this test function's job to test that. Right away, here are the problems:
  1. how to create the conditions for a single outstanding order
  2. how to create the conditions for multiple outstanding orders
  3. how to get the UpdatePlanStatus function to NOT do anything (it updates the DB, and I don't want to touch the DB in this test)
  4. how to get all of this to work without needing real data
This, in fact, is one of the core problems you seem to always run into in unit testing: how to set up your conditions without jumping through a million hoops. Back in the day, I'd have done this:
  1. Go into the database and find an ID in the appropriate table that would give me a single outstanding order
  2. Do the same for multiple outstanding orders
  3. Find an ID for no outstanding orders
Don't worry about the details: the point is that I'd have found data that matched the criteria I was trying to achieve and then, in the unit test, I would have literally used those IDs as arguments to my functions. This would NOT have solved my problem of not wanting to actually run any DB updates. It'd have just gotten the logic to work out. Well... it'd have gotten it to work out until the data in the DB changed. Then, all I'd have is a brittle test. The thing is that you want your tests to work all the time, even a month after you wrote them. Creating tests that depend on certain conditions in the DB itself are a sure way to brittle tests. I know because I've written dozens upon dozens of these kinds of tests. Now then, onto Mocks. For this post, I'm not going to use a mock framework. Just good old fashioned ColdFusion. Here are my goals:
  1. replace the UpdatePlanStatus function inside my object under test with a new version of the function that simply returns the string "updated".
  2. replace the selOutstandingOrdersList function with 3 different functions, depending on what i'm trying to test. In one test, it'll return an empty string. In another test, it'll return a single number. And finally, it will return a comma-delimited list of numbers.
First, my setUp function:
<cffunction name="setUp" returntype="void" access="public" hint="put things here that you want to run before each test">
<cfset pdd = createObject("component","")>
<cfset pdd.init(dsn=application.maindsn)>
<!--- so we can easily mock! --->
<cfset pdd.mixin = mixin>
So we have an object we'll name "pdd". But the fun part here is this: set pdd.mixin = mixin WTF?
    <cffunction name="mixin" access="private">
  <cfargument name="method" type="string" required="true">
  <cfargument name="value" type="any" required="true"/>
   <cfset this[method] = value>
   <cfset variables[method] = value>

There, that's better. I created a new, private function inside my test named "mixin". So, up in setUp, I'm attaching my new function to the pdd object. What mixin does is a tiny bit of CF magic. Mad props to Nathan Strutz, because without this post I'd still be banging my head against the wall. More on that later. To see what it does, let's look at the next chunk of code. A new, mock function:
    <cffunction name="updatePlanStatus" access="private">
   <cfreturn "updated">
It's private so that MXUnit won't run it. All it does is return the string "updated". This means nothing until you look way back up top and see that the function under test makes a call to updatePlanStatus. And that's one of the functions I want to overwrite. So... how to get my new function into my pdd object? Mixin!
    <cffunction name="PlanRollBackShouldFailWithSingleOutstandingOrder" returntype="void" hint="">
    <!--- mock em so we don't touch the DB --->    
    <cfset pdd.mixin("updatePlanStatus",updatePlanStatus)>
    <cfset pdd.mixin("selOutstandingOrdersList",selOutstandingOrdersListOneResult)>
    <cfset s_result = pdd.rollBackFastPlanFromProduction(1,1,1)>
    <cfset debug(s_result.message)>
    <cfset assertFalse(s_result.success,"msg was #s_result.message#")>
    <cfset assertTrue(NOT findNoCase("updated",s_result.message),"#s_result.message# should not contain updated")>
See the call to pdd.mixin("updatePlanStatus",updatePlanStatus)? That's the magic. I'm overwriting the original function with my new braindead mock. This way, I don't touch the DB and I know if that function was run because if it is, it'll return the string "updated". But let's not worry about that right now, because it's honestly not important. The important part is that I've now got a really simple way to do mocking in CF when I need it. Note the next line in the function:
You should now be wondering, "What's selOutstandingOrdersListOneResult"? Here it is, a new private function added to my test case:
    <cffunction name="selOutstandingOrdersListOneResult" access="private">
     <cfreturn 1111>
So, as you see, it just returns a number, 1111 in this case. Thus, in the test above, what I want to ensure is that when the rollbackFastPlanFromProduction function is called, and it calls selOutstandingOrdersList, that that function returns a single-element list. Guess what I do to test multiple orders? If you're thinking "Create a new function that returns a multi-element list, and mixin that function instead... you're catching on nicely.
    <cffunction name="selOutstandingOrdersListTwoResults" access="private">
      <cfreturn "1111,2222">
And then, my new test:
    <cffunction name="PlanRollBackShouldFailWithMultipleOutstandingOrders" returntype="void" hint="">
       <!--- mock em so we don't touch the DB --->       
       <cfset pdd.mixin("updatePlanStatus",updatePlanStatus)>
       <cfset pdd.mixin("selOutstandingOrdersList",selOutstandingOrdersListTwoResults)>
       <cfset s_result = pdd.rollBackFastPlanFromProduction(1,1,1)>
       <cfset debug(s_result.message)>
       <cfset assertFalse(s_result.success,"msg was #s_result.message#")>
Now, I said I'd get back to why I'm thanking Nathan Strutz. I started this test with really simple code I thought would work: set pdd.updatePlanStatus = updatePlanStatus i.e. I had my new, private updatePlanStatus function and I wanted to overwrite the existing version in the object. Pretty simple, right? and when I called pdd.updatePlanStatus() directly, it did indeed return the string "updated". However, when the rollbackFastPlanFromProduction() function called updatePlanStatus, it ran the original code. Normally, I'm pretty good at figgerin' out this stuff. But after a half hour or so, I went all last-resort: I googled it. I don't know how, but I ran into Nathan's post, and he discusses what I honestly didn't know: that when functions call each other internally, they're calling the version of the function that lives in the variables scope. And when you, dear programmer, call a function on an object directly, you're calling it in this scope. So what I did not understand was that functions had two lives, so to speak: a life in variables scope, and a life in this scope. When I was overwriting the function using the direct cfset, I was overwriting it in the external -- this -- scope. But not the internal one. This brings me to the mixin function: the key line is this one:
<cfset variables[method] = value>
This little beaut right here overwrites the internal life of the function, and that's how I can create the mocks. So, thanks Nathan! Now, I leave you with some thoughts:
  1. this mixin business seems pretty handy, and I'd probably want to do it in all kinds of objects. Am I going to have to copy that function into every testcase?
  2. And wouldn't it be nice to NOT have to create those simple little private functions?
  3. And wouldn't it be nice to say "Hey, object, in this case, when you had a single outstanding order, did you NOT run that updatePlanStatus function? And in that other case, where you had no outstanding orders, did you run it?
This is where a framework for creating mock objects becomes valuable. And that's where I'll pick up next. Until next time... happy testing.

Great example of Hard-to-Test Code

Thursday, April 17, 2008

When we talk about designing for testability, it seems pretty abstract. We talk about "encapsulation", like everyone knows what it means. Here's an example of code i just stumbled into that demonstrates not being designed for testability and not being encapsulated Our guest of honor, the function: <cffunction name="getAllUserPlanDocs" hint="returns all plandocs user has permission to" returntype="query" output="false"> <cfset var plandocs=""> <cfset var safename=""> <cfquery name="planDocs" datasource="#dsn#"> select BunchOfStuff FROM planDocuments pd LEFT JOIN PlanDocGroups pdg on pdg.DocGroupID=pd.DocGroupID WHERE pd.PlanDocID IN (#session.user.getUserPlanDocList()#) ORDER BY pd.SortOrder,pd.PlanDocName </cfquery> <cfreturn planDocs> </cffunction> Now, bear in mind, this is probably 3 year old code. I haven't checked source control, but I very likely wrote this code. Who knows? Only source control, and I can't handle the truth right now. So, back then, we weren't doing any unit testing at all. In fact, this is the first time I've written a test for the component to which this function belongs, so I fired that up. <cfcomponent extends="some.parent.testcase"> <cffunction name="setUp"> <super.setUp()> </cffunction> <cffunction name="wizardPlanDocShouldNotBeVisibleUserPlanDoc"> <cfset q_pd= plan.getAllUserPlanDocs()> <cfset debug(q_pd)> </cffunction> </cfcomponent> (Note: I am not an advocate of using a generator to jam out stub tests for existing, legacy code. That's why this is the first test i'm writing for this particular component). All I'm doing here is writing a little test and dumping out the query because I want to see my data first. I just like that approach. I am fully expecting just to get a few-row cfquery dump. So, I go to run this function, and I get an error: "Incorrect syntax near )" WTF? So I open the file from the stack trace view in the plugin, and it takes me to this line in the query: WHERE pd.PlanDocID IN (#session.user.getUserPlanDocList()#) Son of a bitch. This, dear readers, is why people always say "don't use persistent scopes in your code". This violates encapsulation. Because now, the component under test needs to know about some other component in the session scope. Which, obviously, my simple little test doesn't have. So now, my simple little test has to grow, probably by 2x or more, because now i gotta get a session.user object set up and make sure it returns an appropriate list so that line of the query stops failing. But... I'm not going to get mad at myself for writing shitty code 3 years ago. It happens. Here's the lesson, which I'm writing to myself today to remind myself: When you design test-first, or design to make things easier to test, you'd probably not write code like this. Instead, you'd write the function such that it took the User's plandoc list, or maybe even took in a user object. This way, you inject the data you need into the function rather than have the function need to look outside the component for it (in this case, it's looking out into a session scope object). You might be thinking "but if you create the user object to inject in, then that's pretty much the same amount of work, isn't it?" Yes, yes it is. Right now, I'm not at liberty to change this code. Too many touchpoints and I'm not gonna go find them all. So I'm just going to have to make an uglier test case in order to test this functionalty. Coming to think of this, this is actually a good place where mocks are helpful. And I don't mean even using a mock framework or anything like that. I'm talking low-rent ghetto mocks you can use thanks to the glory of CF and mix-in methods. To wit: <cffunction name="wizardPlanDocShouldNotBeVisibleUserPlanDoc"> <!--- function under test uses user.getUserPlanDocList ---> <cfset session.user = createObject("component","")> <!--- I don't want to have to authenticate the user or go through all the rigamarole... so I'm mocking it just trump the getUserPlanDocList function with one I'm creating right inside this test case ---> <cfset session.user.getUserPlanDocList = getUserPlanDocList> <cfset q_pd = request.plan.getAllUserPlanDocs(71)> <cfset debug(q_pd)> </cffunction> <cffunction name="getUserPlanDocList" access="private" hint="I'll use this as a mix-in to mock the getUserPlanDocList function on the user object"> <cfreturn "1,2,3,4,5,6,7,8,9,10"> </cffunction> So here, I just created a new private function inside my testcase that returns a string. I then mix that function into the user object, thereby trumping the one that is in there for real. This way, I can create that static list I would've created anyway to test this code rather than worry about finding a user in the database whose set of permissions, etc will match the conditions I'm trying to recreate. Thank you, ColdFusion. Couldn't have done that in Java. (I know, I know... EZMock). So, my problem is largely solved. When I get the time, I'll get Mike Steele's CFEasyMock set up here at work and I'll swap out my little ghetto mock with a proper mock. Not because it'll change the behavior, but because the intentions will be clearer. It's much easier to udnerstand what I'm doing here if you see "user = CreateMock()......andReturn("1,2,3,4,5,6") " instead of this mix-in approach. There you have it. Testing unencapsulated legacy code, how to avoid such stuff, and how testing could help you avoid this kind of design problem in the first place. Enjoy.

MXUnit at CFUnited 2008

Friday, April 11, 2008

Not only a great excuse to get out of work, walk down the street, and hang out with y'all, but a great way to really learn some cool ColdFusion stuff. Oh yeah, we plan to show off some cool stuff, too. Testing Related Topics: Patterns for ColdFusion Test Automation (Bill Shelton) Automating the build/deployment process with ANT (Marc Esher) CFUnited Registration:
Don't be square. Like, be there.