Sharing your CFEclipse / ColdFusion Builder Snippets across multiple installs

Thursday, December 24, 2009

As I’ve written previously, I rely heavily on snippets when working with CFEclipse. One problem I’ve always encountered, however, is keeping snippets synchronized across multiple Eclipse installations. Typically, this problem manifests when trying to keep them sync’d between my laptop, my work computer, and my PC at home. Recently, this problem has further been exacerbated by ColdFusion Builder. Not only do I want to keep my snippets sync’d between multiple installs of CFEclipse, but I want the same snippets applied to CFBuilder, as well.

Fortunately, this is a very easy to solve problem.

Dropbox

The first step is getting yourself a Dropbox account. Dropbox is a way to share files seamlessly across multiple computers. You’ll want to install dropbox on all your target machines.

Syncing

A) Creating the shared snippets directory in your dropbox

  1. Create a new directory in your dropbox folder, and call it “CFEclipse Snippets” or “CFSnippets” or whatever.
  2. Open CFEclipse (and CFBuilder if applicable).
  3. Go find your existing CFEclipse or CFBuilder extensions. In CFEclipse, you get there by going to window – preferences – CFEclipse. The Snippets install directory field will be there. Copy that path and go to it on your machine. It’ll contain a number of folders and a keycombos.properties file
  4. Cut all of those files and paste them into your new Snippets dropbox directory

B) Pointing CFEclipse to the  new snippets location

  1. If you haven’t done so yet, remove the snippets from the previous CFEclipse snippets location
  2. Open the Snip Tree View in CFEclipse and hit the “refresh” button. Your snip tree view should now be empty since there are no snippets there. This is an important sanity check… don’t skip it.
  3. Copy the path to your dropbox snippets directory into your clipboard
  4. Go to the CFEclipse window – preferences – cfeclipse preference page and paste the new snippets location into your dropbox snippets location
  5. Back in the Snip Tree View, hit the refresh button again. Your snippets should now show up.
  6. From now on, when you add or edit snippets, they’ll be modified in the dropbox location

Other Installs

For all your other installs – different versions of eclipse, different machines, whatever – you just follow the same steps as above from Section B. The bottom line is that if you want to sync snippets, you point all your installs to the same snippets location. Dropbox handles the magic of performing the actual synchronization

CFBuilder

Thus far, I’ve been talking about CFE only. Fortunately, CFBuilder uses the same model and code as CFE. So to get CFB working with the shared snippets, simply open CFBuilder, go to window – preferences – coldfusion – Snippets, and paste your shared snippets directory into that field. Hit Save, go back to CFB, and refresh your snip tree view

Voila… snippets synced!

 

*Note – On some of my Eclipse installs, I had to remove the snippets from the previous/original snippet location before it would recognize the new snippets location. YMMV.

-Marc

MXUnit Eclipse Plugin Updated

Saturday, December 19, 2009

What's new?

  • The pesky “resource is null” problem that some users have experienced when using project-level CFC paths and remote URLs should finally be resolved
  • The “Tag Context” view now has two new menu options: “Copy Exception” and “Copy Tag Context”. This is handy when you need to copy the exception message or the entire tag context, as a string, into a bug report, email, etc.
  • Clicking on an item in the Tag Context view will now trigger the “Compare” Dialog button, if the assertion is comparable (i.e. when an assertEquals() fails). Previously, you needed to click on the failing test to enable the button. That lameness has been resolved.
  • I was watching Terry Ryan’s  video on using the Apptacular ColdFusion Builder Extension to generate MXUnit tests, and I noticed that his MXUnit view in Eclipse had a gigantic “Test History” button. Turns out, I created this as a 32x32 image; Eclipse on Windows resizes this automatically to 16x16, but apparently that’s not true on the Mac. I was aghast at this hideous image, and so now it’s 16x16.

Here are some screenshots of the new stuff in the Tag Context panel:

“Copy” options

tagcontextview_1

“Compare” button enablement

tagcontextview_2

By the way: by default, the text compare dialog shows differences in black/gray. You can change this – as I’ve changed it to a reddish background in this shot – by going to Window – Preferences – General – Appearance – Colors and Fonts – Text Compare – Outgoing change color. Click the “Edit” button, and pick your color.

How do I get the update?

If you already have the plugin installed, then go through the normal update process. On Eclipse 3.5, that’s Help – Install New Software – Work With: MXUnit OR use Help – Check for Updates

If you don’t have the plugin installed, then use Help – Install New Software – Add  and use http://mxunit.org/update as the URL

Enjoy.

--Marc

MXUnit 1.0.8 Released

Friday, November 27, 2009

Los Hombres at MXUnit.org are proud to announce a new smokin' release. Aside from some minor bug fixes, including improvements to the Ant task, and Open Blue Dragon compatibilities (Thanks to Peter Farrell!), we have some sweet features that will make your testing more productive and even downright fun! Ok, that was a stretch ... but check it out anyway... http://mxunit.org/download.cfm

Eclipse Plugin: New data comparison feature. Highlight a failed test and click the little 'book'-looking button in the middle part of the plugin view. It'll pop up a compare dialog and you can use the "next change" and "previous change" buttons on the far right to navigate through the differences.

Note that this will be the last version to support Eclipse 3.3.

Framework: New data driven capabilities via mxunit:dataprovider annotation. Use queries, arrays, and Excel to power your tests.

<cfset name_data = ['joe','taj','mary','juan','hans', 'bjork','chaksa'] /> 

<cffunction name="testNameValidation" mxunit:dataprovider="name_data">
   <cfargument name="name" hint="Each name item in the name_data array" />
   <cfset assertTrue( myObject.validateName(name), "#name# not valid" ) />
</cffunction>


See http://wiki.mxunit.org/display/default/Data+driven+testing+with+MXUnit+dataproviders for details

Note: Application.cfm file is still included and should be removed after install to facilitate integration with frameworks or other applications.

Continuous Integration with Hudson: MXUnit is now being built on a continuous basis. Test results can be viewed here: http://mxunit.org/testresults/index.html and nightly builds can be downloaded from: http://mxunit.org/downloadNightly.cfm

Installation and Default Look and Feel: New logos and layout are incorporated into the default installation.

Also, we'd like to welcome two new MXUnit hombres: Randy Merrill and Bob Silverberg. Randy has been busy doing some much needed project organization and graphics. Bob will be taking on integrating mocking into MXUnit.

Test and Be Happy!
Los Hombres.

This Week in ColdFusion Episode #5

Wednesday, November 25, 2009

The guys put out Episode 5 this week after an an extended break. In this episode, they addressed the question “What processes/practices/tools do you use to help ensure software integrity?”. Brian did most of the talking, and threw around his favorite phrase several times. I can see it now… Brian’s at a cocktail party, the chicks are dancin’, and he’s all “Hey Baby! You got some mad cyclomatic complexity goin’ on!”

OK, OK. All snarkiness aside, this week’s episode touched on three subjects near and dear to me: testing, ANT, and continuous integration. I was thrilled that they spent so much time covering these topics. As they said, each one could fill up a podcast on its own, and I appreciate their honesty here… you can’t just listen to an hour long podcast and be Jonny Test. It takes a long time, and a lot of practice, to get there. Still, the more people out there saying “you need to test, people”, the better. Introductions to the concept, benefits, and resources are never a bad thing, and I think Brian covered these topics as well as can be expected in 60 minutes.

And they gave MXUnit a lot of lovin’ during the show. So, gents, thanks for that! But it’s not gonna get you out of some gentle ribbing. Here goes:

Finger on the Pulse

Your intrepid reporter (uh, me) must confess to some surprise during the first few minutes of the episode. Here are 3 dudes who have been programming in CF for years, who have a frickin’ podcast about CF, but who talked about the CFMeetup almost as if it was some alien thing. Guys… Charlie’s been hosting guest speakers for years on the CFMeetup. The sessions are recorded so you can get them after-the-fact at http://carehart.org/ugtv. This is a tremendously valuable resource for CFers, and I damn near have a hard time believing that people as involved with CF such as yourselves wouldn’t have been on top of that like white on rice.

What’s worrisome to me about this is that if you dudes don’t know about the CFMeetup and carehart.org in general, what’s that mean for the great masses of CF developers who aren’t even as active as you all are, who don’t do a CF podcast? And more importantly, how do we get the word out to more developers about the CFMeetup awesomeness? 

Aside: When I started learning Eclipse plugin development, I struggled with beginner-ish things even after reading a book on the topic. Most newsgroup questions were met with “look at the source”. Fair enough, but… what I really wanted was the answer to “What are the XX most important things a newbie should know about plugin development?”. I wonder how we get the answers to those questions out to new and old CFers alike… it’s not just about code, but also about resources. I wonder how we can all do a better job of connecting developers to the community. We spend so much time worrying about how to evangelize outside the community, but it seems that we need to start thinking about how to connect the people in the community to the excellent available resources that exist that can help make their lives easier. End Aside.

And what about other ways of keeping a finger on the pulse of CF? Having helpful blogs on the radar, knowing helpful people to contact when the going gets rough, etc. I’m in no way suggesting that it’s one’s professional duty to be reading blogs all day, interacting on twitter for hours, attending all the CFMeetups (I rarely do… I catch the ones that interest me at more convenient times, via the recordings) or otherwise engaging in annoying starry-eyed CF fanboi-dom. Simply, I hope that as CF podcasters, you fine fellows can grow into your role as emissaries of our small-but-growing(-at-least-according-to-Adobe) community. When I listen to podcasts for other technologies, I do so because I expect the people behind the mics to be completely in tune with the community. As TWiCF, people are going to expect that of you, and I know you want to deliver.

While I’m heaping un-asked-for responsibilities on you, I think it’s also important that you keep on talking about .NET and the things you are working on outside of CF. Lord knows the CF community is an insular one, and more reasons to lift the head up and look around are welcome.

Get involved. You’ll be better for it, and so will we.

With that ticklish ribbing out of the way, let’s move on.

Unit Testing vs. TDD

Brian spent a lot of time discussing programmatic testing. Good on you, Brian. Clearly he’s a practitioner and knows his stuff. I was also interested in hearing more about the different tools for functional testing. I sure would like to see all these tools in action… the great symphony of testing. How about it, Brian? How about a CFMeetup presentation demo-ing these functional testing tools?

Now, a nit to pick: Brian mentioned once during the episode that there was no difference between writing tests first or last. It’s my experience that there is a tremendous difference between the two. I won’t belabor the point because it’s been done to death (google it), but here’s the bottom line: writing tests first is an act of design. It damn near forces you to write testable code because from the start, you’re thinking, “How can I test this?”. It seems to me that when I write tests first, I end up approaching Uncle Bob’s SOLID principles almost naturally.  However, when I wait till afterwards to test, I miss out on that design opportunity and sometimes find that I have written less-testable code.

I will be the first to admit this is because I am not all that bright, and that other far more capable developers come by easy-to-test code quite naturally, as if the ability were bequeathed unto them by The Almighty Himself.

Furthermore, there is certainly no lack of debate around the desirability of easy-to-test code. More than a few developers believe that writing test first is a crippling exercise, atrophying the brain, because you’re thinking about testability and not the bigger picture of the system. In addition, there are those who believe that thinking about testability inevitably leads to Frankenstein, contorted designs (accessors/mutators where there otherwise wouldn’t be; reliance on non-void functions when if you weren’t testing, the function wouldn’t return void; exposing more methods as public when they should be private, etc). All fair points, and I’ll leave it to bright people to argue over while I go write code.

So: “having tests” (and running them frequently) is helpful for catching regression bugs. Writing tests first is helpful as an act of component design which – to many – leads to higher quality software.

P.S. If you’re interested in other podcasts or videos covering TDD/BDD, Hanselminutes has had a few gems. The interviews with Scott Bellware and Roy Osherove are classics. And you can’t go wrong listening to Joel and Jeff dis TDD and SOLID on the stackoverflow podcast. Finally, in the NSFW category, is Baltimore’s own, the incomparable Brian Liles.

Continuous Integration

I’ve been using the Hudson continuous integration server at work since May and am quite taken with it. I’ve written about it before so won’t belabor the point. But Brian mentioned that he’d like to hear if anyone else out there has any suggestions regarding CI. So… I do. Brian mentioned that having the build server email you on every build becomes a bit much, and I heartily agree. In fact, one of the reasons Hudson impresses me so much is its plugin ecosystem. Specifically, the ‘Extended email plugin’ is outstanding for noise reduction. If you want to read more, have at it.

Other Resources

Brian mentioned ANT several times, so I thought I’d point out Jim Priest’s excellent ANT wiki as a wonderful resource for CFers getting into ANT. In addition, for those interested in learning more about Selenium, Mike Henke has written a decent amount about it. And, finally, check out http://carehart.org/ugtv and search for both ANT and Selenium to find presentations on the topic.

Wait, what? That’s all?

This week’s TWiCF review has turned into less a review of the episode and more just a general meandering about the topics they discussed. Why is that? Well, because I don’t want to be your Cliff’s Notes. Go and download the episode and listen to it. It’s a good one!

Till Next Time

Thanks again, gents, for another enjoyable episode. I’m looking forward to the next one.

Marc, out

Anonymity, Accountability, and Credibility in Blogging

Wednesday, November 18, 2009

One literal definition of anonymity is "without a name". In many contexts I respect people's right to anonymity and privacy if they so choose. However, producing and consuming information is a moral activity [ET], and as such, author identification is critical to the subject's credibility and evolution.
"Publicly attributed authorship indicates to the readers that someone is taking responsibility for the analysis, conversely, the absence of names signals an evasion of responsibility. Readers can follow up and communicate with named sources. Also, names may have reputation for credibility - or not." [ET]

I was motivated to write about this subject when I knowingly engaged in a discussion with an Anonymous commenter on the MXUnit blog. My general rule of thumb is not to engage any Anonymous dialogs, but there was something about this one that caught my attention. Regardless of the reason, I began the discourse, and I am glad I did because it got me thinking, but not so much on the subject of the discussion, which is irrelevant, but rather on the subject of anonymity and credibility. We went back and forth, me and Anonymous. I was looking for proof or citations on some small point and I also wanted to know with whom I was speaking, but I never asked or demanded, which was my failed responsibility. I now conclude that the points the commenter made are 100% invalid. They're invalid not because of their intended or intrinsic merit, they're invalid because Anonymous is not willing to be accountable, and accountability is a precondition for the credibility of information.

This speaks to the broader question of anonymity in the context of the blogging and the web, too. As an information producer or consumer does anonymity do anyone any good at all? Could anonymity possibly do damage? When it comes to discourse, such as expressing ideas on technical issues, today I believe that anonymity doesn't have a place and should be discouraged. So, anonymous comments are now disabled on the MXUnit blog.

I suspect that if you're not willing to take ownership of what you write or say, that maybe you shouldn't say it.  This may be the simple litmus test we should follow.  If we're concerned about backlash maybe the answer is not perpetuating anonymity but creating a stronger culture of acceptance, open mindedness, and freedom.


bill shelton

[ET] Edward Tufte - http://www.edwardtufte.com/tufte/books_be

Data-Driven testing with MXUnit's DataProviders

Wednesday, November 11, 2009

What are DataProviders?

Often in unit testing, you will want to run a test function for multiple inputs. Imagine you have a reformatFullName() that parses a string like “Smith, Bob” into “Bob Smith”. And in your test case, you would like to test your reformatFullName() function for a bunch of different inputs to ensure that you’ve covered common cases and also edge cases.

One common approach to this method is to specify the list/array/whatever – the collection of inputs – in the test function, and then write a loop inside the test function. This is a perfectly reasonable approach, though it does lead to some boilerplate.

DataProviders essentially remove the boilerplate; in other words, DataProviders offer a way for you to specify multiple inputs, but without having to construct the looping. In addition, in MXUnit, you have multiple built-in DataProviders at your disposal, which will be covered in other documentation and blog entries. For now, let’s look at a simple case: an array dataprovider.

How do I use DataProviders in my unit tests?

I Palindrome I

For this example, I wanted to have a little fun, so I chose the classic CS101 problem of Palindromes. I think back to my first programming course – Introduction to Programming in C – and distinctly remember the teacher’s frustrated attempts to drill recursion into our thick skulls. One of the problems he used to demonstrate it was Palindromes, i.e. How do you know if a word is the same forwards and backwards (“mom”, “dad”, etc)?  The teacher used rubber bands; he had us “act out” a palindrome. Oi, poor guy. Anyways…

Before we get into DataProviders, let’s start with a very simple test:

component extends="mxunit.framework.TestCase"{

	function setUp(){
		checker = new PalindromeChecker();		
	}

	function palindromeChecker_Should_ReturnTrue_ForPalindrome(){
		assertTrue(checker.isPalindrome("mom"));
	}

}

That’s a good start, but it doesn’t cover cases where the input isn’t a palindrome, and it sure feels incomplete, doesn’t it? Now, let’s look at a testcase that uses an array DataProvider. In here, we’ll specify an array of palindromes and tell MXUnit: Hey, MXUnit, use this array to run my test, one time for each element in the array:

component extends="mxunit.framework.TestCase"{

	function setUp(){
		checker = new PalindromeChecker();
		//set up dataprovider
		variables.palindromes = ["wow","mom","dad","eye","marccram","bob","poooooooooooop"];		
	}

	/**
	*@mxunit:dataprovider palindromes
	*/
	function palindromeChecker_Should_ReturnTrue_ForPalindromes(String theWord){
		//debug(theWord);
		assertTrue( checker.isPalindrome(theWord), "word #theWord# should have been flagged as a palindrome" );
	}

}

Here, we create an array named “palindromes”. Then, in the test, we specify the custom attribute “@mxunit:dataprovider” and set its value to “palindromes”. MXUnit will see that its been given a dataprovider, see that palindromes is an array, and will run the test one time for each element in the array. One other thing about this test function  should jump out at you: it takes a parameter. Typically, tests return void and accept no params. So why do we need this? Simply, MXUnit will pass the current value in the array into the argument which we’ve specified named “theWord”. If you run this test and uncomment the debug(theWord) statement, then hit ctrl-B on the test case in the MXUnit view in Eclipse, you’ll see each value as the test runs.

sshot-3

And now, let's add a negative test:

component extends="mxunit.framework.TestCase"{

	function setUp(){
		checker = new PalindromeChecker();
		//set up dataproviders
		variables.palindromes = ["wow","mom","dad","eye","marccram","bob","poooooooooooop"];
		variables.notPalindromes = ["woww","momm","marc","eyee","bbobbb","poooeeiop"];
	}
	

	/**
	*@mxunit:dataprovider notpalindromes
	*/
	function palindromeChecker_Should_ReturnFalse_ForNonPalindromes(String theWord){
		assertFalse( checker.isPalindrome(theWord), "word #theWord# should not have been flagged as a palindrome" );
	}

}

Now that we’ve got some tests – and they should be failing b/c we haven’t created a PalindromeChecker.cfc yet – let’s implement the functionality. We’ll do the braindead simplest way first (which is probably the way you would want to do it in real life):

/**
*@hint checks an input string for Palindrome-ness.
*
*/
component{	
	//how I'd program it if this were for a production system
	public boolean function isPalindrome_simple(String theWord){
		return theWord == reverse(theWord);
	}

}

And now let’s have some recursion fun:

/**
*@hint checks an input string for Palindrome-ness.
*
*/
component{
	//use recursion to get it; this is simply to demonstrate recursion
	public boolean function isPalindrome(String theWord){
		//the "base" case... the simplest case we can have that will return true
		if(len(theWord) <= 1) return true;

		/*compare the far left char with the far right char, and then pass the stuff in between
		the far left and far right back into isPalindrome
		 for input: marccram, calls will look thusly:
		   m == m && isPalindrome(arccra) is true
		   	a == a && isPalindrome(rccr)  is true
			 r == r && isPalindrome(cc)   is true
			  c == c && isPalindrome('')   is true
			   len('') <= 1 return true
		*/
		return left(theWord,1) == right(theWord,1) && isPalindrome( mid(theWord,2,len(theWord)-2) );
	}

	//how I'd program it if this were for a production system
	public boolean function isPalindrome_simple(String theWord){
		return theWord eq reverse(theWord);
	}

}

If you run your tests now, you should see them pass. For fun, you could put calls to isPalindrome() and isPalindromeSimple() in each of these tests as a sanity check.

How do I get these DataProviders?

Great question. As of this writing, the DataProvider functionality is in Subversion, and we’re prepping a release. If you just gotta have it right now, but you don’t want to check it out from SVN, let me know and I’ll whip up a nightly build.

Stay tuned for more documentation about the other new DataProviders in MXUnit!

--Marc

Windows 7 CMD Access is Denied

Sunday, November 1, 2009

Access is Denied

Windows 7 installed without a hitch, but I noticed that the bat files I use to turn certain programs on and off no longer worked. Specifically, I have bat files that use “NET START” and “NET STOP” to toggle Windows Services. When I’d run those, I’d get “Access is Denied”. This was particularly annoying b/c I use Launchy to run these bat files to quickly set up and tear down an environment, and this new “security feature” was breakin’ my stride. Here’s how to remind Windows that you are its boss.

Do As I Say, Dammit!

Set Command Prompt to Run As Administrator

  1. From the Start menu, filter on “Command Prompt”
  2. Right click on it, select “Properties”. In the “Shortcut” tab, click the Advanced button.
  3. Check the “Run as Administrator” checkbox
  4. Restart your computer

Here’s a screenshot of what these boxes look like (taken with the new Windows 7 “Snipping Tool”):

commandpromptproperties

It was my experience that you need to restart your computer. When I changed these settings initially, it didn’t change the behavior of my bat files at all. Only after restart did my “Access is Denied” problems go away.

Good luck.

--Marc

How Many Assertions in My Tests?

Friday, October 30, 2009

One Long Test

When I first started writing unit tests with JUnit, I didn't read any books, follow blogs, or otherwise attempt to be the best test writer I could be. I wanted to crank out code, crank out tests, and see the green bar. In fact, my tests looked a lot like this:

public void testGetRequestBeansByIDType() {
    Integer requestID=8;
    Integer statusID=4;
    Integer systemID=1;
    Integer processTypeID=1;
    String inList="0,4,5,6,7,8";
    
    String requestIDSQL = "Select count(*) as NumRequests from " +
        "requests where requestid="+requestID;
    
    String statusIDSQL = "Select count(*) as NumRequests from " +
        "requests where statusid="+statusID;
    
    String systemIDSQL = "Select count(*) as NumRequests from " +
        "requests where systemid="+systemID;
    
    String processIDSQL = "Select count(*) as NumRequests from " +
        "requests where processTypeID="+processTypeID;
    
    String inSQL = "Select count(*) as NumRequests from " +
        "requests where RequestID IN("+inList+")";
    
    try {
        //test requestID
        HashMap<String,Object> row = dbops.getRow(requestIDSQL,1);
        int count = ((Integer)row.get("numrequests")).intValue();
        ArrayList<RequestBean> requests 
            = dbops.getRequestBeansByIDType("RequestID",requestID.toString(),"");
        assertEquals(count,requests.size());
        System.out.println("RequestID rows is " + requests.size());
        
        //test statusID
        HashMap<String,Object> row2 = dbops.getRow(statusIDSQL,1);
        int count2 = ((Integer)row2.get("numrequests")).intValue();
        ArrayList<RequestBean> requests2
            = dbops.getRequestBeansByIDType("StatusID",statusID.toString(),"");
        assertEquals(count2,requests2.size());
        System.out.println("statusID rows is " + requests2.size());
        
        //test systemid
        HashMap<String,Object> row3 = dbops.getRow(systemIDSQL,1);
        int count3 = ((Integer)row3.get("numrequests")).intValue();
        ArrayList<RequestBean> requests3
            = dbops.getRequestBeansByIDType("SystemID",systemID.toString(),"");
        assertEquals(count3,requests3.size());            
        System.out.println("systemID rows is " + requests3.size());
        
        //test ProcessTypeID
        HashMap<String,Object> row4 = dbops.getRow(processIDSQL,1);
        int count4 = ((Integer)row4.get("numrequests")).intValue();
        ArrayList<RequestBean> requests4
            = dbops.getRequestBeansByIDType("ProcessTypeID",processTypeID.toString(),"");
        assertEquals(count4,requests4.size()); 
        System.out.println("ProcessTypeID rows is " + requests4.size());
        
        
        //test IN() statements
        HashMap<String,Object> row5 = dbops.getRow(inSQL,1);
        int count5 = ((Integer)row5.get("numrequests")).intValue();
        ArrayList<RequestBean> requests5
            = dbops.getRequestBeansByIDType("RequestID",inList,"");
        assertEquals(count5,requests5.size());
        System.out.println("IN rows is " + requests5.size());
        
        // test order by....
        ArrayList<RequestBean> requests6
            = dbops.getRequestBeansByIDType("StatusID","3","RequestID Desc");
        RequestBean rb = requests6.get(0);
        RequestBean rb2 = requests6.get(1);
        assertTrue(rb.getRequestID() > rb2.getRequestID());
        
        ArrayList<RequestBean> requests7
        = dbops.getRequestBeansByIDType("StatusID","3","RequestID asc");
        RequestBean rb3 = requests7.get(0);
        RequestBean rb4 = requests7.get(1);
        assertTrue(rb4.getRequestID() > rb3.getRequestID());
    } catch (Exception e) {
       fail("getRow should not have returned an error");
    }
   
}

If you’re looking at that thinking “What’s wrong with that?”, then please read on. If you’re horrified… join the club.

What’s wrong with that?

testGetRequestBeansByIDType, how do I hate thee? Let me count the ways

  1. If some assertion fails up toward the top of this test, none of the other assertions run. So what happens is you get yourself in this constant loop of “it worked last month, now it’s not working. Why?” and so you go fix the failing test, satisfied everything is OK, and then you run em again and another assertion in the same test fails. This infuriates me
  2. Notice how I’m using comments for each of the different tests?  I’m trying to communicate requirements, but I’m doing so in a way that doesn’t much help. If each of these little chunks were separate tests, the test case itself would be much more communicative in my opinion
  3. It’s been my experience that the more things I do in a single test, the more likely I am to create/change some state that modifies the behavior of the subsequent assertions
  4. When developing in this manner, just cranking out tests and not writing descriptive test names that describe the behavior under test, I’m missing out on one of unit testing’s greatest benefits: using it as an aide to clarify thinking. This is a huge topic, and one that I don’t want to get into here. Suffice it to say, anyone who’s done TDD for a while knows exactly what I mean. Anyone who hasn’t done it will only know what I mean once you get yourself in the camp of the people who have done it. You have to earn this answer

So what’s a boy to do?  Personally, I’ve become fond of BDD-style naming. I prefer long, descriptive test names; I prefer as few assertions as necessary in a single test.  This leads to some additional lines of code, but I use snippets for this so I’m not spending any more time writing more tests than I would be if I were simply writing more assertions. The resultant clarity is worth it.

Ahhhh, That’s Better

Here’s a more recent example:

<!--- basic tests to make sure the 'outer' logic is right--->
<cffunction name="recipientsWithAffectedLetterCode_Should_GetPhoneNumberCoordinates" returntype="void" access="public">
	<cfset var coordinates = "">
	<cfloop list="#affectedLetterCodes#" index="code">			
		<cfset coordinates = getCoordinatesWithWCRBCD("JH89",code)>	
		<cfset assertTrue( StructCount(coordinates) GT 0,"Coordinates struct should not have been empty for #code# but was." )>
	</cfloop>
</cffunction>

<cffunction name="recipientsWithoutAffectedLetterCode_ShouldNot_GetPhoneNumberCoordinates" returntype="void" >			
	<cfset var coordinates = "">
	<cfloop list="#unaffectedLetterCodes#" index="code">
		<cfset coordinates = getCoordinatesWithWCRBCD("junk",code)>	
		<cfset assertTrue( StructIsEmpty(coordinates),"Coordinates struct should have been empty for #code# but was not. It had keys: #StructKeyList(coordinates)#" )>
	</cfloop>
</cffunction>

<!--- more granular tests for the specifics of which number to get;
yes, this duplicates the logic in the production code. So be it. --->
<cffunction name="WCRBCD_JH89_gets_2011_PhoneNumber" output="false" access="public" returntype="any" hint="">
	<cfset var coordinates = getCoordinatesWithWCRBCD("JH89")>
	<cfset assertEquals("866.420.2011",coordinates.PhoneNumber,"")>		
</cffunction>

<cffunction name="WCRBCD_MK01_gets_9446_PhoneNumber" output="false" access="public" returntype="any" hint="">
	<cfset var coordinates = getCoordinatesWithWCRBCD("MK01")>
	<cfset assertEquals("866.356.9446",coordinates.PhoneNumber,"")>	
</cffunction>

<cffunction name="WCRBCD_NotMK01AndNotJH89_gets_7436_PhoneNumber" output="false" access="public" returntype="any" hint="">
	<cfset var coordinates = getCoordinatesWithWCRBCD("NOTMK01")>
	<cfset assertEquals("866.281.7436",coordinates.PhoneNumber,"")>	
</cffunction>

Ignore for a minute the weirdo codes (WCRBCD, JH89, etc) as they are domain-specific and well-known for people working on the project. The point here is that the names describe exactly the various expected behaviors of this single function under test (getCoordinatesWithWCRBCD()). When one test fails, I don’t have to spend time reasoning about whether previous logic or function calls are the culprit. I and other team members don’t have to learn about the behavior by reading comments because the function names themselves are the documentation.  I don’t have that “Fixed-this-assertion-and-now-another-one-is-failing” problem. If something breaks, I know very soon because we run everything in a continuous integration environment. Code changes, tests run, something breaks: I can go right into Hudson and see exactly the behavior that used to work and now no longer works. If multiple things break as a result of a code change, I see those multiple things, not just the first assertion that failed.

Is there a downside to this approach? Yes… if you’re typing out each function by hand, i.e. < c f f u …, then you’re going to be typing more. But if you’re typing out functions by hand, you’re doing it wrong! MXUnit comes bundled with a bunch of useful Eclipse snippets. Read the documentation, install them, and use them. Then, when you want to add a new test function, you type the word “test”, hit CTRL-J, and up pops this box:

testbox

From there, you type in your test name. For example: “thisFunction_should_DoFancyThings” and hit Enter. This code gets added to your editor:

test_result

Typing problem solved.

The Bottom Line

In the end, the fewer assertions you put into your tests  -- as opposed to fewer tests with more assertions -- the more descriptive, stable, and design-for-testability-helpful your tests will become.

Looping over an array in CFScript

Thursday, October 29, 2009

The other day, I was searching the cf9 docs for an equivalent of the <cfloop array=”#myarray#”> functionality introduced in CF8, and it appears that was not introduced. So it was back to ArrayLen() looping for me. Tonight, I saw TWiCF’s Micky Dionisio put this out on the Twitters, and I thought it was cool:

“#coldfusion 9 tip -grab a handle on the iterator of an array by calling someArray.iterator(). then u can use that in script FOR/WHILE loops!"

I never thought of that, so I wanted to give it a whirl.

<cfscript>
//pre-cf8
list = "one,two,three,four";
a = listToArray(list);
for(i=1; i LTE ArrayLen(a); i=i+1){
	writeoutput(a[i]);
}

//traditionally, with cf8+ syntax
a = ["one","two","three","four"];
for(i=1; i <= ArrayLen(a); i++){
	writeoutput(a[i]);
}

//with an iterator
//uncomment if you want to see what's available: writedump(a.iterator());
iter = a.iterator();
while(iter.hasNext()){
	el = iter.next();
	writeOutput(el);
}
</cfscript>

Warning: don’t try to skip the iter = a.iterator() and el = iter.next() steps in an effort to save a few lines of code! If you try to loop while(a.iterator().hasNext()), you’ll get stuck in an endless loop because you’ll be returning a new iterator each time. And each time you call iter.next(), you’re advancing through the array; thus, doing writeOutput(iter.next()); doSomethingWithThisElement(iter.next()); will give you very different results from what you’re expecting. You’ve been warned.

Thanks for the tip, Micky!

This Week in ColdFusion Episode #3


I posted my review of TWiCF episode #4 yesterday, and today on the drive home I listened to episode #3. The first half of the episode was a discussion of conferences, since the episode was recorded at the same time that Adobe Max 2009 was underway; the second half focused on the question “To Flex or not to Flex”, which basically means “Should I spend my time learning Flex, or should I learn a rich javascript framework instead?”. Or, perhaps it also means “If I were building a project that required X, Y, and Z, what does Flex give me, and what does <insert fancy JS framework here> give me, and why should I choose one or the other for my project?”. 

Conferences

The line of inquiry for the first part was “Is it worth spending the money to go to a ColdFusion conference?” I can picture some listeners thinking, “What? Are you mad? That’s the stupidest question I’ve ever heard”. But I have to say: I think it’s a completely valid question. Mike brought up his experience with conferences and I think he describes a feeling that a lot of people encounter: that feeling of “I’m here to bask in the glory of the presenter”… to be talked to, to be lectured. To soak up their wisdom and be grateful for this brief glimpse of their brilliance. Lo, a star!

I’d bet that more people than are willing to admit have had those Stuart Smalley moments. “Am I worthy?” “Am I good enough to be here?” Perhaps. Perhaps not. Regardless, I can understand the sentiment.

I want to say to the TWiCF guys – and to any listeners out there with whom those thoughts resonate – you will not find a more inclusive, caring, friendly, willing-to-engage community than the ColdFusion community. And I’m not just saying that as a guy who’s fairly outgoing. I attended cfObjective for the first time in 2008, and I was so grateful for the warmth and inclusion that I saw amongst people whom I consider heroes: Andy Powell, Simeon Bateman, Peter Bell, Brian Kotek, Bob Silverberg, etc. These people rock! And they love to talk about this stuff with people, no matter whether you’re a constant blogger or just a guy/girl who wants to learn some CF and possibly something beyond CF that maybe, just maybe, will become useful down the road. In fact, if I’m reading the pulse of these people correctly, I’d bet they’d be just as or more passionate about talking to people who really, really want to engage than to some super-smart know-it-all blowhard. And you get these small, intimate BOF sessions where it’s you and a handful of super awesome people simply talking about a topic you’re all interested in. Seriously: how often do you get to sit in a room – as peers – with Barney Boisvert, Joshua Frankamp, Laura Arguello, Oguz Demirkapi, Luis Majano, and no-names like me talking about Git in a session led by Peter and Sim? And it’s not because you’re a Git superstar, it’s because you just want to learn more about it face-to-face?  These are the reasons you pester your bosses to go to conferences.

This year’s CFUnited was outstanding, as well. The content was superb. The speakers were everywhere. It’s so easy to spot a dude who you saw speaking a few hours earlier, walk up to them, and start firing questions. Interestingly enough: one of the strangest takeaways for me from CFUnited this year didn’t happen as a result of me sitting in a session or picking some speaker’s brain. Rather, it came from a conversation I had after my session on automation with Sumit Verma. He asked me if I had ever heard of a certain Windows command for working with Services, and I hadn’t. So I emailed him afterwards, we chatted, and now what I learned from Sumit – a guy who wasn’t speaking but who was in my session – has had a profound implication for something I’m doing at work right now. How sweet is that? Teachers and Students? Speaker and attendee? Nah. The blurrier that line, for me, the better. It’s people who know things interacting with people who know other things. That’s where magic happens.

You always hear “You get out of it what you put into it”. It’s true. When you go to these conferences, try to engage. Walk up to people and say Hi. Even if you don’t like hanging out in the hotel bar, go sit down anyway. Order a coke. Eat peanuts. Whatever. Just open yourself up to people. When some funny-lookin’ bald dude says “yo, you wanna go hang out with us and smoke a stogie?”, say “yeah, sure”, even if you don’t smoke stogies. You don’t have to suck down a Cohiba! Grab your beer or your coke and you might be hanging out with Terry Ryan and Ryan Stewart and Adam Lehman, talking awesome stuff about what’s coming up in CF and the Flash Platform.

Notice how I spent three paragraphs talking about stuff that wasn’t sessions?  If the value from 5 pm to 12:00 am is this good, imagine how good the stuff from 9 to 5 is! Brian said that for one of the conferences he attended, he was “required” to give a debriefing to his team when he returned to work. I can’t recommend this enough, for several reasons:

  1. It encourages you to become a more active audience member. Rather than being a passive mouth-drooler, you’re more likely to at least take notes during sessions
  2. It provides you an opportunity to revisit the material afterwards. Consider: you’re going to go to one of these conferences, sit through 15-20 sessions, and no way will you remember it all. Going back through your notes – several times – gives you more time to think and interact with the material
  3. Your team members benefit from the time you spent at the conference. Surely it’s not the same as being there. But maybe something you say during your talk will inspire them to go check something out themselves (“What’s this CouchDB thing? Sounds interesting. I’m gonna go play with it”). Who knows, maybe you’ll present on something you learned at a conference, it’ll intrigue one of your team members, and you might collaborate on a small project down the road where previously you’d have going it alone.

I strongly believe that if you attend either cfObjective or CFUnited in 2010, it will be worth your while. Each conference has a great variety of tracks: ColdFusion, Flex, Air, Project Management, etc. Each conference is run by people passionate about what they’re doing. Each conference brings in wonderful speakers. Each conference LOVEs to get new speakers (Brian Carr… I’m talking to you! Suck it up… we need solid presentations on real-world ReST with CF).

The guys also discussed presentation formats other than the traditional “eyes forward” style. Dudes… I hear you. I don’t know what CFUnited has in store for 2010, but I can say that the people behind cfObjective are taking this very, very seriously. This will be a shake-up year. This will be a year when engagement is paramount.

XJS?

The whole way through the 2nd half of the episode, the guys talked about “XJS”. I was like “WTF is XJS?” Is this some new javascript framework I didn’t know about? They didn’t mention CF8 at all, so I wasn’t sure if they were talking about ExtJS, which is used for a lot of the CF8 Ajax widgetry. I was cornfused, as we say in Pennsyltucky. A look at the website shownotes afterwards cleared it up. They were talking about ExtJS. So my question is: How DO you pronounce it? Since I was introduced to EXT in 2007, I’ve always said “E-X-T”. Can I get a ruling from the line judge on how to pronounce this?

LCDS, push/pull, comet, and CF9

There are probably a dozen people in the world who haven’t been confused by “what is LiveCycle? what is LCDS? what is BlazeDS? What can you do with Flex and XXXDS? and with ColdFusion? What DS hoogie comes bundled with CF, and what are its license restrictions?”  I know I surely was confused by this. I think I have a handle on it now though. Bottom line: if you’re running CF8, and you want Push technology, you can use the “honor license” LCDS that comes bundled with CF8, and you can also use BlazeDS (free, open source). Apparently LCDS will scale beyond more than a few hundred simultaneous users, though I do not have confirmation or links for that. Also, I believe CF9 now comes bundled with BlazeDS, so you needn’t worry about licensing for your push/pull applications. In the real world, what this should mean is that you can drop messages into your Blaze-backed gateway, have them pushed out to all your Flex clients registered on that endpoint, and you’re good to go.

If you’re creating a 20,000-user online Flex-based push/pull poker game, well, you’re probably looking at some LiveCycle Data Services licensing (note: this does NOT mean you’re spending 250k on a LiveCycle ES license. LiveCycle DataSevices != LiveCycle ES).

If I’m one of the hapless masses  who is still putting out incorrect information, please, someone, set us all straight.

WAAAAAA, BOOOOOO

Mike started off the episode with a baby crying sound effect that almost caused me to wreck my car. Egads, man, that was too real. But funny nonetheless… pulled Brian right into that one. At the end, Mike pulled Micky into another gag, too, which I thought was pretty funny.  I hope Micky lets the air out of Mike’s tires.

 

Till Next Time…

Thanks again, gents, for enjoyable company on the commute home. Till next time…

Marc, out

This Week in ColdFusion Episode #4

Wednesday, October 28, 2009

The Skinny

I commute about 45 miles each way from Pennsylvania to the Baltimore burbs. Generally, I am not a fan of this commute. However, as one who likes turning the proverbial lemons into lemonade, I’ve found that podiobooks and podcasts are a great way to spend the 90+ minutes a day in the car. I try out all different kinds of podcasts, and more often than not I delete them from my iTunes after a few episodes. With the new TWiCF podcast, I think I’ve found a keeper.

Episode 4 of This Week in ColdFusion (TWiCF) was released on October 19. I had seen mention of this new CF podcast a few weeks back, and it came across my radar today, when I was in front of the computer with easy access to iTunes. So I subscribed and listened to this latest episode on commute home from work.

In short, I like it! It’s relatively quick paced, the guys have good podcast voices, and the content is fun and informative.

While listening, however, I wished I had a good way to take notes because I wanted to speak to a few items. Rather than hijack their comments thread, I figured I’d use this blog instead. Beats hearing about unit testing, don’t it?

Without further adieu, some points I’d like to make:

ColdFusion Builder Pricing

Where did Mike get that $250 number? I have it on good sources from my inside-Adobe operatives that the price is somewhere between “We ain’t telling you” and “STFU and quit asking”. (My quotes). So, please, don’t believe any numbers anyone says about CFB until you hear it from CFB’s PM, Adam Lehman.

Debugging in Eclipse

Let there be no question: you can use Eclipse, right now, for free, and do step debugging on your CFML. Provided you’re running CF8, all you need is the ColdFusion 8.01 Eclipse extensions. Once installed, it’s a matter of a 2-minute setup and then off you go. For an outstanding introduction to using the debugger, see Charlie Arehart’s CFMeetup presentation. While I’m at it… check UGTV often. It’s a veritable gold mine of CF content.

You can set “watchpoints” as well so you don’t have to scroll through gazillions of variables. I will say, though, that I personally hate being in a debugger. I figure if I’m in a step debugger, then either my app is to complicated or my unit tests are not helping me. Just sayin’ is all.

So, key words: CF8, Eclipse, Free, Easy.

Full CFScript and CFEclipse

The guys talked a bit about the enhanced cfscript support in CF9. In particular, the fact that you can write components without any tags at all:

component {
 public Order function init(required numeric id, array items = ArrayNew(1), String shippingMode="StandardShipping" ){
  StructAppend(variables,arguments);
  return this;
 }

 public void function dumpVars(){
  writeDump(variables);
 }
}

The problem in CFEclipse, as they mentioned, is that these files look like you’re editing in notepad when you open them in CFE. Good point, guys. I’ve emailed the CFE dev group to see what the level of effort would be to get the syntax highlighting working correctly.

Snippets WishList

One of the guys – I think it was Micky – mentioned a feature he’d like to see in Snippets, based on his work with Visual Studio. Essentially, he described the ability to type a trigger, and then have the cursor “follow” you as you “tab through” so you could type in what you wanted to type. I believe the Eclipse equivalent of this is functionality called “Templates”. Before I talk about templates and CFEclipse, though, let’s talk about what you get with the Snippets that already exist.

You can already get this behavior, or at least a “pretty good” version, in snippets as they’re currently implemented. I’ve found that for the (way too few) people who use snippets, this is something they don’t know about. I’m talking about “Variables”, and they are the cats pajamas. I wrote about this a while back, so I encourage you to spend 10 minutes sharpening the saw to save yourself hours cutting.

Now, onto Templates. Fortunately, Denny Valliant recently added an initial implementation of this feature into CFEclipse. Unfortunately, the CFEclipse team didn’t publicize it at all and so no one really knows about it. So it’s time to talk about them. Please read this post to learn about this sweet new feature with lots of potential for time-saving awesomeness.

The REST Discussion

I thoroughly enjoyed this part of the show. I’d like to see Brian give a presentation – with loads of examples, in particular from the Producer point of view – on designing and publishing REST-ful services. It’s easy to consume them… but how about architecting an application that produces the services? What strategies exist? What tools/frameworks? What big problems have you encountered? What if I want to do something more than just delete User 5? Instead I want to place an order for 13 copies of 5 different books and have them all mailed to my relatives in the 4 corners of the globe (OK, extreme, but you get the picture… simple scenarios = suck… I want more depth)?  What do you say, Brian?

Oh… and how do you unit test them? This is a blog with “unit” in its name, so I’d be remiss If I didn’t mention that.

What was Missing?

What was missing from the This Week in ColdFusion Podcast, Episode #4? Why… “this week in ColdFusion”, of course. I guess when I started listening, I was expecting more of a TWiT format, or perhaps at least more newscasty-ness to it. It is entitled, after all, “This week in…”. Consequently I expected more news items. I’d really like to have at least some time during the newscast devoted to keeping we CFMLers up to date on what’s happening in the community. News items, new projects, interesting blog posts, etc. Personally, I rely heavily on the Java Posse podcast to keep me up to date on all the things in the Java and Scala world that I don’t have time to investigate myself. Adding more news-i-ness into the TWiCF podcast would be a huge win for me.

Till Next Time…

Again, it’s encouraging to see another CF Podcast, especially one that appears to have a lot of promise. I hope Mike, Micky, and Brian can keep it up. I can only imagine how grueling it must be to produce a podcast, so kudos to all members of our community who devote their time to such endeavors.

I hope each episode is engaging enough to make me want to come home and write a review / response like this one.

Marc, out.

New in CFEclipse: Templates


This is part of an ongoing series on Timesavers. The goal is simple: short, easily-digestible posts designed to help developers get faster and more productive

The recent 4th episode of This Week in ColdFusion inspired me to finally get the word out there regarding this sweet new feature of CFEclipse: Templates. This has been in CFEclipse since June, but we never advertised it or blogged about it. If you’re a java developer who uses Eclipse, you probably rely on templates so heavily that you’d think they were just baked right into the IDE. You know how when you’re editing java, and you type “for”, and it pops up a bunch of suggestions? That’s what this is.

Denny Valliant’s initial implementation only came with two default templates, though I suspect more will come down the road. It’s easy to write your own, though, and here’s how:

  1. Window – Preferences – CFEclipse – Editor – Templates
  2. In the Templates window, hit “Add New”
  3. Type in a name, a description, and your templates. You can use “Variables” by using ${} syntax, and the contents in between the brackets can be any arbitrary name. Trust me, you’ll see how this works in a minute
  4. If you want custom stuff – like where to put the cursor or some other “standard” variables, you can insert them with the dropdown list
  5. Hit OK and get out of the preferences window. The window looks like this:

cfe_templates_1

Now, in your editor, I created a new array, went down a line, and started typing the characters that begin with my new template name. For example, in my screenshot, I had a template named “looparray”, and so I started typing “loop”, and in my editor, that template appeared:

cfe_templates_2

I hit “Enter” to accept the template, and now I’m in the template, and with each tab, I’ll move on to the next part of the template.

cfe_templates_3

In this example, I have an array named “stogies”, so I tell my template that my array is named “stogies”, and it replaces my “template variable”. I want my index variable to be named “stogie”, so I type that in, and it replaces it in the cfloop tag AND in the block of code that followed.

cfe_templates_4

This can save you a lot time right now, for free. This has the potential to be a huge timesaver, especially if developers write their own templates and share them with the community via the “export” functionality.  Hopefully, as the CFEclipse developers get more time, more templates will come bundled with CFEclipse.

Viva la TimeSavers!

Stackoverflow Dev Days D.C. 2009 Review


At $99 and just down the street, it was hard not to go. The State Theatre is a very cool venue, a restored old theatre turned into Rock club. In Falls Church, VA, my wife's home town, it was just about as close as close could be for me. I sat front row like I was at concert or something, and, as I was reading Twitter on my G1, I noted a tweet by Liz Frederick from Adobe who was also at Dev Days and sitting right behind me. Cool! It was nice to meet her, we had a pleasant chat, and I learned some things about how Adobe is reaching out to developers.

Executive Summary
: It was totally worth it for this local guy. I left wanting to buy a Mac, do iPhone development, manage my projects with FogBugz 7 (and figure out WTF bayesian statistics are), get more serious with Python, finish my Google App Engine projects, and front end all my web apps with jQuery. I met some super smart, nice, interesting people, had a good time, and left inspired. One cool thing was that most of the speakers were local folks. Not sure about Bruce Eckel, but the others were all from the general D.C. area. I think this was a good thing to do for everyone involved - the conference, the attendees, the speakers, and the community.






Here're my notes:

Joel Spolsky Opening Keynote - Great warm up video - I got a real good laugh right off the bat. Great slides. Joel is a skilled and entertaining presenter. Joel's keynote was about Simplicity vs. Features, with specific emphasis on usability. Spend that extra time doing things for the user. Don't interrupt them for stuff your app should be doing. Dialogue Boxes == bad. User's don't care how much work you did to make the app; be humble enough not to show off your effort. Hide the details of how it was built. Get out of the way and strive for elegance. Elegance always looks like it should be more than it is. He showed some an example Python spell check app that I remember seeing earlier this year - it was like 150 lines of code or something - spell checking in 150 lines?! Elegance. I get it. Thx.

Oh... here's a grin. I'm sitting in the front row and Joel is making a point about usability and he points to me and says, "Pick a gender!" ... huh? Me? ... "Yeah. Just say 'male' or 'female'" Me: "uh ... male". Then he goes into a story about "Bill" who's having an affair with his secretary who wants to blackmail him for $100K while Bill's wife is walking down the hall getting ready to bust them and Bill needs to cut a $100K check with QuickBooks but doesn't have the time to answer the stupid dialogue about upgrading the software because Bill's wife is about to open the door on the two lovers. Then Joel must have realized my name just might be Bill, so he asks, "Is your name, Bill?". Well, yeah, not that you could read my scribbled name tag in a dark room from 20 feet away ... But Joel's a pro and rolls with it no problem. I wasn't bothered by any of this at all, by the way. I was stoked to be out of the office around smart people.  I guess he remembered me later, because again he gestured towards me and used "Bill" to exemplify an idea. Ok, now if I can just figure out how to get some cool product front row to buyers ...

Dan Pilone : iPhone  - Dan says Objective C is not as bad as people say. It takes Apple 2 weeks to approve/disapprove an app, and they will test it and make sure it complies with rules and regulations. Even if it's been approved for release 1 and they missed something, like a minor licensing violation, it can get kicked out again in future releases. In short, you need to plan on this 2 weeks and dot your T's. There's money to be made but you have a short window of time to grab it and leverage the Apple store presence. There's a boat load of iPhone customers, so, lots of opportunity, but also lots of competition (like 100K apps in the iPhone Store) and it's growing rapidly. You can do the $0.99 app, which needs volume sales to succeed, of course, or your can go with the $20 app, which has less sales but might be more stable.

Scott Allen ASP.NET-MVC - Scott is clearly a great coder. Anyone who can write code on stage in front of 200-300 of probably the smartest developers in D.C. and beyond gets a big hand from me. This is me holding up my lighter for an encore - more! I don't code in C#/.NET but if I get the opportunity, I can only hope to be as skilled as Scott. And I will go right towards the MVC framework he described. One thing I personally would have liked to see, which he mentioned, was unit testing. Apparently, MVC generates test stubs and allows you to test in some interesting ways, like, defining a "context". I'll need to look into it, but I sensed an interesting approach to testing.

Joel Spolsky on Fogbugz 7 - I had no idea it did all those things - much much more than issue tracking. Evidence based scheduling ... bayesian something or other statistical intelligent learning (like I have any idea) with cool graphs ... wow. They're integrating their own version of Mercurial, too, which looks like Git from the command line, and it integrates well into FogBugz with code review capabilities and task management. Having an enterprise tool like this available where I don't have to administer it, just makes sense. I'd rather focus on getting my project organized and tasks completed vs. that AND learning to administer the beast. The usability that was emphasized in FogBugz is very attractive. Joel said he wants it to be as easy to enter tasks as typing in Notepad, and he have some good demonstrations. It also supports and seems to be geared towards Scrum.

I also "got" the idea of Stack Exchange as a way to capture domain specific knowledge. Joel's view is that the corporate wiki can fail because it's document centric and experts don't really want to write documents nobody reads. However, with a question-based environment, like Stack Exchange, specific questions can be answered and aggregated into a knowledge base. I like that idea. It's more like pull rather than push.

Lunch - Cheap box lunch & fine by me. Groups for various discussions were set up during lunch - agile, free software, start ups, etc... I wanted some sunshine and the seating did not really lend itself to round table discussions, for me. So, I sat outside in the sun, walked around the block, and talked with a PHP developer from Chicago on the corner of Washington Blvd.

Bruce Eckel Python
- Bruce talked about why Python makes sense and how the language develops vs. how Java/C/C++ evolved. Basically, it can do what Java/C/C++ do but with a lot less effort and do it dynamically. Also, the fact the Python is community driven and not the product of a company, makes it an appealing choice. He's working on a version on Thinking in Python, only it's under a different title. The interesting thing about this book is that it will be community driven. It sounds like an interesting idea.

Bruce is a forward thinker, to say the least, and when pinged he talked in abstract terms about what he sees as part of the future - human computer interaction and machine symbiosis. Not in a sci-fi machines taking over the world kinda way, but we are clearly dependent on machines today, and the future will be focusing on how this interface grows. I agree. Typing a keyboard is one weird way to communicate for a human. Speaking and audio with visual cues is how we, as humans, do it.

I think that might be Mr. Hankey's brother hugging the python.


The last time I heard Bruce was in 1998 at my first ever tech conference - SD '98. I worked as a volunteer so I could get free admission and one of my assignments was to support Bruce's Thinking in Java talk. I was so excited at the time when he said, "Hey, can someone go get some audio tech help?" I was on it, man ;-)   


Jonathan Blocksom Google App Engine  - I think everyone could identify with Jonathans' down-to-earth style. As humble as he was, it was clear to me that he is very smart ... Google smart. This was the one topic that was a review for me. I had covered all this with some App Engine prototypes I did over the Summer. But Jonathan discussed the AppEngine architecture, quota/pricing model, and how to get up and running. It was very cool that Jonathan could do an entire talk about distributed applications and not once use the term "Cloud". Good job.

Richard D. Worth , JQuery - A very well organized and detailed presentation. One of the main things I took away was a simple phrase about JQuery architecture: "Find Something, and Do Stuff". I love JQuery and it's been my goto for about a year, yet I never saw it like that. But it makes perfect sense. Richard clearly is passionate about JQuery and is willing to teach. He just created a poll (Poll Daddy) for a 1 or 2 day JQuery talk in D.C. in Nove,ber or December, so, keep an eye out. It will be well worth it. I'll be there.


Wrap up: There were a lot of people who flew in from out of town and came from more than an hour away. I felt fortunate that I could wake up and drive 20-30 minutes to do a cool 1-day conference. What strikes most about this, and it's not Dev Days specific, but it's about the concept of a cheap 1-day event that's on the road, vs. a multi day conference in some place like Vegas. I kinda like the idea of 1-day, but I'm not sure if I would do a road trip for just 1 day, maybe that's what surprised me about the number of folks from out of town. I spoke with a PHP developer who flew in from Chicago. I also connected with a Python developer originally from Northern Ireland and living in Philly. Chatting with Liz Frederick from Adobe was also a breath of fresh air. So, for this developer, getting out is important. But for me,I question whether or not the multi-day big events with evening schmooz-fests are the best environment for learning. They're fun, for sure, but I sense a better way to learn new stuff ...

It's funny, but I posed this question on Stakoverflow ... whether or not micro conferences like Dev Days are a good place to do peer-2-peer learning, and the Stackoverflow zealots quickly shut my question down, closed it, as "Non Programming Related". Whatever... So, I'll ask it here: How do you learn best and grow, and what inspires you?

“Look, Ma. No SQL!” – MongoDB and ColdFusion Part 3

Sunday, October 25, 2009

Addendum: The code examples below were built on earlier versions of MongoDB and will not work with 1.6+. Also, use Marc Esher's Github fork for the most current version of CFMongoDB.


Querying MongoDB documents with JavaScript and ColdFusion

In the first part of this series I presented my fundamental attraction to MongoDB, and in the last post I talked a bit about some of MongoDB's core concepts. In this part, I'll explore some approaches, challenges and a proposal for reading data stored in MongoDB.  The source code (still in a proof of concept state) is available at Github - feel free to fork it and make comments.

MongoDB's data is persisted as binary form of JSON (BSON) and the admin engine language is JavaScript. (We'll get to ColdFusion shortly) Let's work with the Blog document structure - JSON- presented in Part 1:

{ "_id" :  ObjectId( "65c5b8782745e04afc2abf00"), 
  "AUTHOR" : "bill_1" , 
  "TAGS" : ["Comics","Games","Python","NoSQL","Ruby"] , 
  "PUB_DATE" : "Thu Oct 22 2009 07:42:19 GMT-0400 (EDT)" 
  "BODY" : "Lorem ipsum dolor sit amet ..." , 
  "TS" : 1256177399955 , 
  "TITLE" : "Blog Title No.1 ..." 
}; 


The example code generates 1000 such documents. Imagine 1000 items with the author and title values incremented for each one from 0-999.

From MongoDB's admin console, you can query this structure like this:

> db.blog.find()

... which returns all the documents in the blog collection and is semantically equivalent to select * from blog

Here's some other basic examples for querying the blog collection. This should give you a feel for how to get data back from MongoDB:

Retrieve a blog entry by ID:
SQL: select * from blog where id = '65c5b8782745e04afc2abf00'
MongoDB: db.blog.find( {_id: ObjectId( "65c5b8782745e04afc2abf00")} )


Retrieve 10 of the most recent titles:
SQL: select top 10 * from blog order by pub_date desc
MongoDB: db.blog.find().sort( {PUB_DATE:-1} ).limit(10)

Retrieve 10 entries by a given author:
SQL: select top 10 * from blog where author = 'bill_1'
MongoDB: db.blog.find( {author: 'bill_1'} ).limit(10)


To retrieve data from MongoDB you use JavaScript and object notation. An interesting point is the syntax for search criteria. The find(...) method expects a JavaScript object with the criteria; e.g., {author: 'bill_1'}. For basic equality checks you pass in the name of the field you're searching and the value. To do comparative tests you can use the syntax:  db.collection.find( { "field" : { $gt: value } } ) for greater than. There's almost a full range of operators available, including full regular expression searches - Wahoo! Take a quick peak at the docs and come back ... http://www.mongodb.org/display/DOCS/Advanced+Queries . What do you think? That's the command line shell. Very cool. But how can we do that in a ColdFusion webapp? At the root, you wrap ColdFusion around the Java client:


//setup - no necessary for every search 
//create a mongo instance 
mongo = createObject('java', 'com.mongodb.Mongo').init( server_name , server_port ); 
//get a handle to the database 
db = mongo.getDb(db_name); 
//get or create a collection (e.g., Blog)   
collection = db.getCollection(collection_name); 

//build search criteria 
criteria = createObject('java', 'com.mongodb.BasicDBObject').init('TITLE','bill_1.*'); //regular expression 
//execute search 
results = collection.find(criteria); 


It's helpful to note that the primary interface for both writing data and querying data is com.mongodb.DBObject. This extends java.util.HashMap and represents a map of name value pairs that are saved to the datastore and also used as search criteria. In the example above note that you create a BasicDBObject with a name and value. This object is to the find method and used to query the datastor.

As for the syntax, I'd prefer something different. So, I here’s my attempt at a DSL in ColdFusion which looks like this:

results = mongo.collection('blog').startsWith('TITLE','Blog Title No.60').search();

This returns all documents whose title starts with the second argument. This is semantically the same as select * from blog where title LIKE ‘Blog Title No.60%'.

For other string related criteria there’s also endsWith(field,value), exists(field,value), regex(field,expression) and others. Since this is a DSL we can chain these together to better express a search:

results = mongo.collection(‘blog’).
                exists(‘BODY, ‘MongoDB’).
                eq(‘AUTHOR’, ‘bill’).
                after(‘PUB_DATE’,’09/12/2009’).
                search();

This is asking for all the blog entries by bill that were posted after September 12, 2009, with Mongo in the body. Now, I think this is clear, but maybe you don’t. That’s cool.  Here’s the proposed syntax. Again, this is all a proof of concept, so feel free to comment.


   Proposed DSL for MongoDB searches:  
  results =  mongo.collection('blog').   //optionally set the collection to search, otherwise it will use current collection 
             startsWith('name','foo').   //string 
             endsWith('title','bar').    //string 
             exists('field','value').    //string 
             regex('field','value').     //string 
             before('field', 'value').   //date 
             after('field', 'value').    //date 
             $eq('field','value').       //numeric 
             $gt('field','value').       //numeric 
             $gte('field','value').      //numeric 
             $lte('field','value').      //numeric 
             $in('field','value').       //array 
             $nin('field','value').      //array 
             search('title,author,date', limit); 

Proposed:    search( keys=list_of_keys_to_return, limit=num, start=num, sort={field=direction} );



The string and date searches all work as expected, but there are issues with the numerics and the conversion from ColdFusion to Java representations. Numeric comparisons need to support all numeric types. The numeric comparison methods are all prefixes with $. This is done to match the MongoDB JavaScript syntax and to get around ColdFusion's reserved keywords. The array searches have been proven, but need work.

One last goodie: $where(…) : You can also do adhoc queries with JavaScript like so:

results = mongo.collection(‘blog’).$where( ‘this.TITLE == ‘Blog Title No.1 || this.AUTHOR == ‘bill’ ’).search();

Important notes:
  • When using $where, you must prefix the item you are seaching for with 'this', which corresponds to the current collection.
  • MongoDB mentions that there is a greater perfomance demand for $where
  • Searches are case sensitive. This may throw some folks, but it might help to think in terms of JavaScript, which is case sensitive.
  • Case sensitivity applies to both keys and values. And note, too, that ColdFusion creates uppercase keys when creating structs, unless you create the struct using this syntax: my_struct["my_key"] = my_value;
  • There is no built in boolean OR, yet, except in $where

Feedback is welcome, of course.


Up Next: Storing Blobs, Indexes, Admin 101


Test and be Happy!

RESOLVED: VMWare 6.5.x Stopped Working on Ubuntu 9.04

Friday, October 23, 2009

Symptoms first occured after upgrading from 8.10 to 9.04 and included:
  • Start VMWare Workstation from menu: Nothing appears to happen. No errors. 
  • Start VMWare Workstation from command line and get the following error

billy@mxunit-ubuntu:~$ sudo /usr/bin/vmware
Logging to /tmp/vmware-root/setup-25257.log
modinfo: could not find module vmmon       
modinfo: could not find module vmnet       
modinfo: could not find module vmblock     
modinfo: could not find module vmci        
modinfo: could not find module vsock       
modinfo: could not find module vmmon       
modinfo: could not find module vmnet       
modinfo: could not find module vmblock     
modinfo: could not find module vmci        
modinfo: could not find module vsock       
/usr/bin/vmware: line 31: 25257 Segmentation fault      "$BINDIR"/vmware-modconfig --appname="VMware Workstation" --icon="vmware-workstation"    


Resolution:

billy@mxunit-ubuntu:~$sudo mv /usr/lib/vmware/modules/binary /usr/lib/vmware/modules/binary.old
billy@mxunit-ubuntu:~$sudo vmware
Solution from : http://ubuntuforums.org/archive/index.php/t-1212220.html



"Look, Ma. No SQL!" - MongoDB and ColdFusion - Part 2

Tuesday, October 20, 2009

Addendum: The code examples below were built on earlier versions of MongoDB and will not work with 1.6+. Also, use Marc Esher's Github fork for the most current version of CFMongoDB.



MongoDB Core Concepts

In the last post, I started to scratch the surface of MongoDB and focused on some of its efficiencies. I mentioned that querying would be next, but I think it might be better to briefly discuss some of MongoDB's core concepts first. Note that some of these will also apply to other schemaless datastores, like CouchDB and Google's BigTable

The biggest challenge getting your head around a schemaless datastore is shifting in how you think about data persistence. If you're like me, you've spent most of your professional career thinking in relational terms, and maybe have even formally studied set theory, and know what tuples are and who E.F. Codd is. The concepts of tables, rows, relationships, normalization, and structured query language (SQL) might be so ingrained, that it's truly difficult to think of data storage any other way. If you want what you got, don't change what you do. But, if you sense that there might be a better way to store and retrieve some kinds of data, you're certainly not alone.

Many of the schemaless/schema-free terms and concepts are either inspired by, or are borrowed from the relational model, yet there are fundamental differences - I'll touch on a couple of the differences later. For now, let's look at some of MongoDB's concepts and how they might relate to a relational mindset.

  • Database - Like in a relational model, this is a logical and physical storage area for data, which are Collections of Documents. A MongoDB server can have multiple databases. 
  • Collection - Similar to a table, a Collection is a group of Documents. 
  • Document - Similar to a row in a table, a Document represents an instance of some object in a Collection. In CFML, a Document object is a structure (or an array). A Document can contain other documents by either directly embedding them (see below) or by reference. 
  • Indexes - Like a relational database index, this is a Document attribute registered with MongoDB for query performance and query optimization. 
  • Queries and Cursors - Lastly, like a relational database, you query Collections. The query results are common programming data structures: Iterators or Arrays, but are referred to more generally as Cursors.

Example CFML/Java Implementation:

<cfscript> 
 person = {
  name = 'bill',
  profession = 'programmer',
  age = 'older than dirt'
 }

 mongo = createObject('java', 'com.mongodb.Mongo').init();

 db = mongo.getDb('my_db'); 
 collection = db.getCollection('people');
 doc = createObject('java', 'com.mongodb.BasicDBObject').init();
 doc.putAll(person);
 collection.insert(doc); //correction! thanks to marc e.
 criteria = createObject('java', 'com.mongodb.BasicDBObject').init('NAME','bill');
 query = collection.find(criteria);
 while(query.hasNext()){
  item = query.next();
  dump( item );
 }
</cfscript>

Note that the "get" operations above will create the object if it does not exist. This a true example of schema-freeness

Embedded Document Example:
This is much simpler than the name might imply. All you need to do nest one struct within another:


person = {
 name = 'bill',
 profession = 'programmer',
 age = 'older than dirt',
 address ={
   street = 'main st.',
   city = 'anytown',
   planet = {
     name = 'Earth',
     planetary_code = '0xFFB3540A'
   }
 }
}

If you've worked with XML, the nested document structure should be familiar. With MongoDB, Document objects can nest an arbitrary number of other documents. Check out the Schema Design guidelines for some ideas.

Key Differences:
From a developer's perspective, the key differences are in how you think about the data and how the data is retrieved. Instead of thinking about your data as a set of related 2-dimensional tables with rows and cells, you are free to think about data in terms of object structure or composition (hence the name Document). Storing the data is pretty trivial in MongoDB, and quite a relief compared to writing a complex insert statement. Retrieving data, though, is where stumbling might occur, and big picture consideration should be given when embarking on a new project and considering RDBMS or schemaless. Some people have recommended that the general guideline for deciding whether or not to use a schemaless datastore is to know whether or not your data is high volume and low value. In other words, if you have very complex data, such as financial data that needs to be frequently aggregated at a very atomic level (high value), or if your data not so complex and maybe, too, you have a lot of it (low value,high volume) e.g., a Blog. In the case of the former, maybe a relational database might be better; in the latter, maybe MongoDB.

As for querying the data, this is what you might be used to:

select  p.*, a.*
 from Person p inner join Address a on p.id = a.pid
 where p.id = 'some_id'

In MongoDB, given the person Document above, this would be done more like this:

person = db.people.find( {_id: some_id} ).limit(1);


Both say the same thing, semantically: "Given a person's ID, return the person and their address"

Up next, for real, Querying MongoDB in detail.

"Look, Ma. No SQL!" - MongoDB and ColdFusion - Part 1

Monday, October 19, 2009

Addendum: The code examples below were built on earlier versions of MongoDB and will not work with 1.6+. Also, use Marc Esher's Github fork for the most current version of CFMongoDB.



From the perspective of a developer who prefers to think in terms of real-world objects, relational databases tend to slow me down. I'm not making some broad sweeping statement that "databases are bad" - they're not. They're just another layer of disparate technology that I need to contend with. What if we could just write code and store and retrieve those objects quickly? ORM and caching technologies do this well and they're aim is integration with relational databases, and Adobe has nicely integrated Hibernate into ColdFusion 9.


Alternatively, there's been some very interesting work done in the area of NoSQL, or schemaless data persistence. These efforts provide relatively small, fast, and distributed key-value data stores, and allow applications to store and retrieve objects in an efficient manner - and programmers don't have to worry about creating and managing tables or schemas. These projects include (among others) CouchDB, Tokyo Tyrant, and MongoDB. In short, these can be viewed as efficient and huMongous persistent hash tables. I chose to work with MongoDB simply because I found it the fastest to get up and running on both Unix and Windows.


Quick example usage (full CFML source available here):

<cfscript>
  blog = structNew();
  blog.title = '';
  blog.text = '';
  blog.author = '';
</cfscript>

This is a very basic data structure (a CFML structure) representing a blog entity. We know what we would need to do to write this to a database - create the table(s), specify columns and datatypes, and write SQL to insert, update, delete, and fetch. Maybe we'll use stored procedures or DAOs, too. With MongoDB here's a simple example implementation - a proof of concept API:

<cfscript>
  mongo = createObject('component','MongoDB');
  //create the blog
  blog = structNew();
  blog.title = 'Look, Ma. No SQL! MongoDB and ColdFusion';
  blog.text = 'Rapid development with MongoDB ...';
  blog.author = 'bill shelton';
  mongo.put(blog);
 </cfscript>  

That's it - the entire structure is now persisted. For the jQuery and JavaScript folks, you're really going to like this. The stored data format (BSON - binary JSON) in string representation is plain old JSON and the MongoDB admin console is a JavaScript shell:

>db.default_collection.findOne()
{"_id": ObjectId( "58c5b878f16bdb4a94f4d300") , "AUTHOR" : "bill shelton" ,
"TEXT" :"Rapid development with MongoDB ..." , 
"TITLE" : "Look, Ma. No SQL! MongoDB and ColdFusion"}

How cool is that?

Adding additional fields or data. There's no need to create new tables or columns -  MongoDB takes care of this for you. If they don't exist they'll be created. Just add the elements you need to your code. Let's add some tags and comments to our blog entity. Note that these are not simple string properties - this makes the example a bit more interesting and shows how powerful MongoDB can be.

<cfscript>
  mongo = createObject('component','MongoDB');
  //create the blog
  blog = structNew();
  blog.title = 'Look, Ma. No SQL! MongoDB and ColdFusion';
  blog.text = 'Rapid development with MongoDB';
  blog.author = 'bill shelton';
  blog.tags = ['cool stuff', 'data persistence', 'hadoop'];
  blog.comments = [
    comment = {
      author = 'anonymous',
      comment_text = 'you suck at unit testing'
    }
  ];
  mongo.put(blog);
 </cfscript>  

Here we added an array of tags and array of structs ... comments. Again, our entity is saved to the data store. Let's read this back out and print it:

<cfscript>

 mongo = createObject('component','MongoDB');

 blog = mongo.findOne();// get the most recent entry
 writeoutout(blog.title);
 writeoutout(blog.author);
 writeoutout(blog.text);
 writeoutput(blog.comments[1].author);
 writeoutput(blog.comments[1].comments_text);

</cfscript>

Here's a basic example of how to update and delete data.

<cfscript>
  blog = mongo.findOne();// get's the most recent

  blog.title = 'My New Blog Title';
  mongo.update(blog);
  //if we want to delete it: 
  mongo.delete(blog);
</cfscript>

For someone who has embedded many lines of  SQL in Java and CFML and knows the pain with keeping all the parts in synch, this is very appealing.  Note the dot notation for traversing, too - It's familiar and fast. Also, my goal with all this as not to do a straight port of the Java driver. Rather, I tried for something extremely simple and fast for writing and fetching CFML structures to and from MongoDB datastores.

Though one of the big motivators for MongoDB seems to be geared towards creating very large distributed data stores, from a simple rapid development perspective, it's downright sexy.

Getting Started with MongoDB and ColdFusion
1. Download and start MongoDB - this is pretty much just downloading, unzipping, and running ./mongod. But, you might have to create a directory for the data. It's well documented and I was able to get it up and running in a couple of minutes on both Windows and Ubunutu.
2. Download the Java driver and put it in your ColdFusion server's classpath or use Mark Mandel's JavaLoader.
3. Check out the example code and the MongoDB wrapper for ColdFusion.

Note that there's a lot of other information about all this from both the developer's perspective and the admin perspective. I recommend reading up on use cases and schema design, as well as the other docs.

Next up: MongoDB query syntax .
 
Test and be Happy!