Comments on MXUnit Blog: A Fluent Paradigm For Refactoring Conditional Logi...

Formulate the arguments symbolically and test its ...

2009-11-11T04:56:09.462-05:00

Formulate the arguments symbolically and test its validity using the truth table. Also mention critical row (or rows).

If I study hard or I get rich, then I get A’s.
I get A’s.
 If I don’t study hard, then I get rich.

Where p = I study hard,
q = I get A’s,
r = I get rich.

@manfred, doh! my bad. the print() method is a hac...

2009-01-23T14:44:00.000-05:00

@manfred, doh! my bad. the print() method is a hack i threw into the framework - i'm always adding stuff i think should be in coldfusion anyway. i'll send up another zip file with the correction. just replace that with debug() and you'll be good to go. thanks for point it out!

bill

Billy, thanks for the quick response. Yes, that wa...

2009-01-23T12:10:00.000-05:00

Billy, thanks for the quick response. Yes, that was the problem. I was trying to figure out what your code was trying to create on line 88, as I have never tried/seen that syntax.
FYI, I am running CFMX v8,0,0,176276 Enterprise. I will try it on CFMX v8.01 server to see what happens.

All of the tests in /validator/ValidatorTest.cfc?testMethod pass except resetGoodieShouldClearExceptionArray(). It indicates the variable PRINT is undefined (line 5 of ValidatorTest.cfc). Is that a function that you use? Or is that in MXunit? I was just trying to figure out today if I have the latest version of MXUnit.

Thanks,
Manfred

@manfred, thanks! It looks like, for some reason,...

2009-01-23T11:21:00.000-05:00

@manfred, thanks! It looks like, for some reason, your CF install does not like my implicit struct notation. I just re-downloaded it myself and ran it ok on my windows xp machine (CF8.1 multi-server). To make the tests pass, you might want to change line 88: args = {...}; to something like:
args = structNew();
args.foo = 'bar';
args.mydate = now();
a = arrayNew(1);
a[1] = 123;
args.theArray = a;
...
or some such. The args data is pretty much arbitrary, but it's used to test complex data.

Try that and see if that works.

best,
bill

This is great. Thanks. I downloaded the .zip file ...

2009-01-23T11:02:00.000-05:00

This is great. Thanks. I downloaded the .zip file and placed it in my wwwroot, but I am getting a CFML compiler. I am running CFMX v8 and MXUnit. When I run http://localhost//validator/ValidatorTest.cfc?method=runTestRemote
I get this error:

The CFML compiler was processing:

* A script statement beginning with args on line 88, column 4.
* A script statement beginning with function on line 86, column 3.
* A cfscript tag beginning on line 2, column 2.

The error occurred in D:\WebSites\validator\ValidatorTest.cfc: line 88

86 : function setUp(){
87 : validator = createObject("component","validator.Validator");
88 : args = {args = [1,2,'asd',now(),{foo=['bar',-123.09932,[123,'00']]}]};
89 : }
90 :

@marc, re:dataprovider, I wasn't thinking too much...

2009-01-22T05:24:00.000-05:00

@marc, re:dataprovider, I wasn't thinking too much about generating inputs, but I think you got the picture. Imagine a test signature like this:

[cffunction name="testThatLoopsOverAQuery" dataprovider="someQuery"]
[cfargument name="rowName1"]
[cfargument name="rowName2"]
assert (rowName1 gt rowName2, 'or whatever - #someQuery.currentRow#');
[/cffunction]

Note parameters. This test would be executed for each row of the query. We should also be able to pass in arrays and structs.

re:covering the truth table - I didn't think about the connection between the dataprovider and truth table coverage until you brought it up. I think I like that idea :-). Note to self: As per @marc esher, see how we can leverage a dataprovider annotation and automatic truth table generation.

bill

bill, are you talking about auto-generating inputs...

2009-01-22T04:41:00.000-05:00

bill, are you talking about auto-generating inputs for a dataprovider annotation? or just providing the dataprovider annotation and the mechanism for running it:

[cffunction name=myFunctionSHouldDoThis mxunit:dataprovider=someFunctionNameThatReturnsAQuery mxunit:failfast=false]

it sounds like you're talking about generating meaningful inputs... that seems awfully sketchy to me. but then, i'm no big thinker.

i've only read the docs on testng's dataprovider, but from what i've read, i'm thinking mxunit will be able to go even further with the concept, largely because you can so easily do dynamic invocations. you could have pre and post functions to run before and after each row of the dataprovider is executed, and then a post-dataprovider function that acted on the results of the full loop. this owuld just be gravy if you needed it; otherwise, with no functions being passed in, mxunit would simply loop over the test function itself once for each row in the dataprovider, execute the test, and behave accordingly based on the failfast attribute.

great stuff here. i am looking forward to the day where test writing becomes more about thinking about inputs and less thinking about boilerplate junk

@marc, thinking again about covering the truth tab...

2009-01-21T04:46:00.000-05:00

@marc, thinking again about covering the truth table ... there should be a recursive algorithm that, given a set of values intended for boolean expressions, traverses or visits each combination. I suspect that we could generate a minimal structure of valid and invalid values and then visit each combination. I'll bang this out after cup #2 ;-), and after the adobe articles, and after the AOP widget, and after this that and the other ...

@marc, that's smart testing! It's also one of the ...

2009-01-16T11:41:00.000-05:00

@marc, that's smart testing! It's also one of the TestNG approaches I want us to build into mxunit - dataprovider.

@bill, I see that now. Good stuff!

2009-01-16T11:27:00.000-05:00

@bill, I see that now. Good stuff!

Thinking out loud here, and maybe this is just stu...

2009-01-16T10:35:00.000-05:00

Thinking out loud here, and maybe this is just stupid, but when I think of truth table logic, I'll often write tests that use.... tables.

I create a private function in the test, and within that, i'll use cfquerysim to generate my table.

then, the test itself will loop over that test, run my functionUnderTest with each of the inputs corresponding to a single row in the table.

If something happens I don't expect to happen, then I'll append that to a structure. I do this so that the test does NOT fail fast... I want every scenario to be run.

Then, at the end of the test, I'll write an assertion on that resultant structure.

Here's a quickie example of what I'm talking about. This is thoroughly untested!

[cffunction name="usernameShouldFailForSpecifiedReason"]
[cfset var table = createValidatorTruthTable()]
[cfset results = StructNew()]
[cfloop query="table"]

[cfif table.ExpectedException eq ""]
[cfset tmp = obj.createLogin(table.username,table.password)]
[cfif NOT tmp.success]
[cfset results["#username#_#password#_ShouldNotFailButDid"]]
[/cfif]
[cfelse]
[cftry]
[cfset tmp = obj.createLogin(table.username,table.password)]
[cfthrow message="#ExpectedException#_ShouldFailButDidNot"]
[cfcatch type="#ExpectedException#"][!--- do nothing... that's what we want ---][/cfcatch]
[cfcatch]
[cfset results["#username#_#password#_#cfcatch.message#"]]
[/cfcatch]
[/cftry]

[/cfif]

[/cfloop]
[cfset debug(results)]
[cfset assertEquals(0,ArrayLen(results),"The following combinations failed: #StructKeyList(results)#")]

[/cffunction]

[cffunction name="createValidatorTruthTable" access="private"]

[cfset var q = ""]
[cfoutput]
[cf_querysim]
q
username|password|ExpectedException
mesher|easy|TooShortException
mesher|l33thx04||
mesher|nopunct|nopunctuation
........
[/cf_querysim]
[/cfoutput]
[cfreturn q]

[/cffunction]

@Oscar, good point. In general, I feel that there ...

2009-01-16T10:09:00.000-05:00

@Oscar, good point. In general, I feel that there is no good solution to reducing the "amount" of tests here or elsewhere. We can, however, create tests that are clearer, more targeted, and easier to maintain. This approach really is aimed at breaking down the complexity of conditional logic into smaller more manageable pieces. A real-world value, I see, is when the production code is executed and fails, you will know exactly what failed and why. You could then display that info back to the user.

thanks,
bill

I see how this refactoring makes for a much better...

2009-01-16T09:41:00.000-05:00

I see how this refactoring makes for a much better code and is way easier to read, but the part that I'm missing is that I don't see how the use of the validator object will cut down on the test cases you need to check. The different constraints are still there and your test code still needs to test each one of those conditions (and their combinations). So if the idea is to reduce the testing I'm not really seeing how that is achieved. Can you please clarify that?

@Ilya ...I, too, now have an additional 10K SLOC t...

2009-01-16T07:51:00.000-05:00

@Ilya ...I, too, now have an additional 10K SLOC to refactor ;-) thanks! bill

Makes me want to go and immediately rewrite a ton ...

2009-01-15T22:23:00.000-05:00

Makes me want to go and immediately rewrite a ton of my code. This looks very elegant. Thanks for a great post.