A new version of CFMongoDB is available on the developer branch, here: https://github.com/marcesher/cfmongodb

This version includes the latest MongDB Java driver as well as 2 significant changes:

1) The new MongoClient is now preferred over the legacy Mongo object. All of the unit tests and examples have been updated to use this object. It's API compatible with Mongo. The significant difference between MongoClient and Mongo is that MongoClient now waits for the database to acknowledge writes, rather than tossing it over the socket and returning immediately. 10Gen wrote about this change here: http://blog.mongodb.org/post/36666163412/introducing-mongoclient. In essence this simply changes the default WriteConcern from UNACKNOWLEDGED to ACKNOWLEDGED.

In addition, I've enhanced the API so that setting MongoOptions is now much simpler. In fact, reverting to the previous default of UNACKNOWLEDGED write behavior is a very simple affair, if you choose that. Please see examples/mongoClientOptions/index.cfm for usage.

2) Integration of the recently introduced MongoDB aggregation framework... see http://docs.mongodb.org/manual/aggregation/. It's in general much faster, and easier to use, than MapReduce. 

The Aggregation Framework is now a first-class citizen in CFMongoDB, just as normal searching and MapReduce are. Please see examples/aggregation/aggregate.cfm for usage.

Huge thanks to Sean Daniels for starting this work!

Full Disclosure:  though all unit tests and examples work as expected, I am not currently using either of these in production.

If you hit any snags, have suggestions, or otherwise want to improve or contribute, please file issues and pull requests.

Happy Coding (and testing),

Marc

MXUnit 2.1.2 has been released. This is a small, single-feature patch release which affects the behavior of beforeTests() and afterTests() when run from within the MXUnit Eclipse plugin. Previously, these would behave just as setUp() and tearDown() when run from within the plugin, in that that they would run on every single test method. Now, they run once per component, as expected.

Get the latest bits here: https://github.com/mxunit/mxunit/downloads

--Marc

ColdFusion 10 recently shipped… congratulations to the Engineers, Product Managers, Evangelists, and testers. Great work. It’s time now to start planning for CF 11. Herein I offer my unsolicited advice… a wish list, if you will.

tl;dr

• I want to code in Groovy
• I want safety by default
• I want a Grails-like framework
• I want Package management (plugins)
• I want GPars or Akka concurrency
• I want a console

Now… let’s dig in. Warning: many words will be sacrificed in the writing of this post. I care about this stuff.

What I love about CF

I started with CF over 10 years ago. During my first programming class (Intro to C), a woman showed me a website she had built using this thing called ColdFusion. It had awesome Dynamic-Drive style JavaScript flyout menus. If you remember back to 2000, you know you HAD TO HAVE flyout menus. That night, I bought the CF 3.1 WACK from half.com. The book and CD arrived a few days later. I’m flipping through the index trying to find the “menus” section. Wait, what? WTF is SQL? What’s a query? Where in the hell are the javascript menus? What an idiot I was.

A week or so later I had built my first data-driven website.

That’s why I love CF. It was a gateway to my next career; it’s been extraordinarily good to me. I’ve met some of the best humans on the planet… in the ColdFusion community. I’ve contributed significantly to open source CF projects. I’ve built apps that have brought value into this world. Thank you, ColdFusion.

For me, CF has never been about “language”, but about features. Historically, it’s made harder things simple… Data access, web services, XML processing, file manipulation, Email, FTP, VFS, PDF creation, etc. More recently we have ORM, ehcache, Websockets, etc. All great stuff.

Community members have given us a slew of frameworks, libraries, applications, etc. to use. We suffer an embarrassment of riches in this regard.

And yet….

We can do better

The Language

CF’s strength has always been in its features, but never in the language itself. To follow along, you will need to separate, in your mind, the features you use from the language with which you use them.

I recently spent @3 months on a project with Groovy and Grails. Groovy was fairly popular a few years back but has lost some mindshare on the JVM in recent years, largely due to the ascendance of Scala and to some degree Clojure. Lots of reasons exist for this, which I won’t get into. Suffice to say, of these 3 languages, Groovy is probably the most natural transition for CF developers. It is a fantastic language, to be sure.

On this project, I am still shocked at how little code I typically had to write, compared with CF code that would’ve done the same thing. But I’m not going to fall into that trap of “see, it’s fewer characters and thus SUPERIOR!”. Instead, I’ll say that the language is simply more joyful. It’s a nebulous thing, open to much finger wagging… I know this. I haven’t quite yet identified for myself why it’s more joyful to me. Probably it’s a mix of Collections functions, collections looping, closures, mixing static and dynamic typing, sane concurrency abstractions, and so forth. Plain old every day stuff like array and map manipulation, object population… bread and butter stuff that pays the bills.

I want to use Groovy in my ColdFusion applications. I want to create CFCs that are, in fact, all Groovy.

In addition, when I need to use CFML, I want the following

• Null-safety operator, and the rest of the Groovy operators
• Death to the var scope
• Death to CFQueryParam

Better operators. Go check out Groovy’s operators, specifically the “Safe Navigation” operator. If you’ve used ORM in ColdFusion, you no doubt have isNull() checks littered throughout your code. It is, without doubt, the absolute worst part about moving from a query to an ORM based persistence model. Nulls have caused me more pain than virtually anything in CF save for the var scope (more on that little bastard momentarily). Groovy solves that problem with the safe navigation operator. In my CF views, I must write this constantly: <cfif not isNull(object.getSomeThing())> #object.getSomeThing()#</cfif>. In Groovy, you’d simply do this: user?.address?.street

Death to the var scope. ColdFusion components are unsafe by default. Don’t believe me? Create a CFC. Add a function. In that function, use cfquery. Store that CFC in the application scope (it’s a DAO after all, so make it a singleton).

If you don’t see a problem with what you’ve just done… that is the problem!. Because what you’ve just done is perfectly sensible. No sane server-side language should allow variables to be in the global scope by default. In a function, the default scope should be local, period. There is no compromise here. Local. Default. Period.

In CF 11, I want the var scope to be unnecessary. Old code that uses “var” won’t break, because it’ll simply become a no-op keyword to be deprecated. It will have no effect. Any variables created in a function will be local scoped. If you wish to create it in the variables scope, you do it intentionally, not accidentally.

Death to cfqueryparam. The hell you say! Hear me out. OK, maybe not the “death”, per se, but how about it being there for when you absolutely, positively need it but otherwise you don’t use it much day to day. Here’s why:

When I do this: <cfquery>select * from foo where id = ‘#form.Uberl33tH4x0rPwninMyA55#’</cfquery>

I am now vulnerable to sql injection. So I must use cfqueryparam to escape it. But why?

I propose that CF should turn variables into bound parameters in queries. It is, after all, a variable. CF knows this. So do what cfqueryparam does and turn it into a bound parameter for me. Again: safety by default. By the way, Groovy does this.

Better Java datatype interoperability. CFMongoDB does some trickery to turn booleans into booleans, and numbers into numbers, before stuffing data into Mongo. It’s because “1” is not 1, and “true” is not true, and when you query against fields containing numbers and booleans in Mongo, you darn well better make sure your numeric data are numeric and your boolean data are boolean; otherwise, you’re be left wondering why collection.find({age : 5}) won’t return anything, even though you know damn well you have a 5 year old in your collection.

The problem is that CF treats numbers and booleans as strings, and consequently when they’re inserted into Mongo as part of a document (think: struct), or used in a query (which is also a document), they’re usually (not always… GRRR) inserted as strings. Thus, you must use javacast or in the case of CFMongoDB, something faster and more clever.

I do not want to worry about this datatype business any more. In a struct, a number is a number, and a boolean is a boolean. No compromise. That’s it. Number. Boolean. Period.

Enough about language. Moving on..

Grails-like functionality

Grails is a web framework, inspired by Rails. It’d be a mistake to say it’s merely a clone, however.

I don’t want to get into whether CF should ship with an “official” framework. I have no skin in that game and want no part of that argument. Instead, I’ll simply put out there what joy Grails has brought me (and what pain). CF has always seemed to me to be an interesting mix of language and framework, so perhaps these things can be brought into CF without the need for a framework.

Dynamic finders. Grails’ dynamic finders are inspired by Rails. Basically, if you have an object User, and it has properties firstName and DateOfBirth, you can query for instances by doing this: User.findByFirstNameAndDateOfBirth()…. for all possible combinations of properties. It’s brilliant.

ORM, in general. Go read the grails docs on establishing many-to-one and one-to-many relationships in a grails domain model. Compare that with the latest code you had to write in a CFC to do the same. Night/Day.

Object population. One feature I’ve come to rely on heavily is Grails bean population via a Map. This is Groovy based but Grails adds some sugar on top. Given a Map (struct), you can create a new object like this: def project = new Project( yourMap ). For all the keys in that struct, it’ll call the corresponding setters from your object. In addition, if you use a “.id” idiom, it’ll also do relationship lookups for you: If your Project has a “region” property which is a Region object, and your map looks thusly: myMap.region.id = 5, then calling new Project( myMap ) will cause grails to go look up the Region object with id “5”. Joy.

Plugins. Most modern web stacks have a plugin system… but not CF. Rails has gems. Node has NPM. Grails has plugins. Basically, the problem it solves is that it makes it much easier to bring other people’s libraries into your application. Want to use Spring Security in Grails? How about this: >grails install-plugin spring-security-core.

That’s it.

Want a pre-built user admin interface? >grails install-plugin-spring-security-ui

I will absolutely admit that my experience with grails plugins has been love-hate. Incompatibilities, missing maven repos, inscrutable class-loading bugs. This is likely the nature of plugin systems in general, and I don’t have a solution. Still, overall, having a solid plugin system is preferable to none at all. This is a no-brainer.

Grails-inspired. I want the CF team to spend a month or two building an application in Grails. Not a blog. Build something meaningful… some kind of internal application. Whatever. Just use it and learn it. Now, take everything that is awesome, and put it into CF. Think of what could be even awesomer, and put that into CF. Then, think about the things that suck, and make sure they don’t end up in CF.

Yes, I know a lot of those things are possible in CF with a bunch of onMissingMethod hackery. That’s not the point. The point is I want it baked into CF.

Concurrency

I created cfconcurrent because concurrency is hard, CFThread is suitable for a small set of use cases (fire-and-forget), and superior solutions have existed for a very, very long time. Still, even java.util.concurrent, which cfconcurrent exposes, is considered by some very smart people to be too low level even today. They believe even better abstractions can exist on top of it. Enter GPars and Akka. GPars is Groovy’s concurrency solution, while Akka is Scala-based but also exposes a Java API. I will do neither of them justice here. Instead, I’ll simply state my wish:

I want CF to include one of these libraries as its official concurrency mechanism

Console

Groovy ships with a “console” and a REPL (read-eval-print-loop). The console is a GUI that you pop up and write code in. The REPL is a command prompt that executes your code. I don’t care so much about the REPL, but the console is gold.

Grails also ships with a console that is based on Groovy’s console. It extends it by providing you access to your grails application – all your objects, models, plugins, etc. It’s brilliant.

On this aforementioned Groovy/Grails project, my workflow changed considerably. Typically, I write a bit of code, along with a test, and ping-pong between code and test. However, because testing is this weird mix of awesome and horrifying in Grails (out of scope for this discussion), I found myself writing a lot more code early on, in the console, and once it worked as I wanted I’d bring it into my app and start testing it. I know, bad TDD’er! Whatever. When integration tests take 10, 20, 30 seconds just to start, this is the kind of thing you do to meet deadlines.

The console makes it so easy to simply play. Because it’s simple text, and your only debugger is println, I found myself keeping things very simple. In addition,  you can easily inspect objects with its inspector, which eliminates the need for a proper debugger.

Even now, for every day scripting tasks, I typically use the groovy console.

If you’ve ever used Chrome’s (or Firebug’s) JavaScript console for fiddling with your JS, or even if you’ve used JSFiddle, then you’ll know what I’m talking about and perhaps can understand why it’d be fantastic to have something like this for ColdFusion.

I want a console. I want to open up a CF Console, type code, play, explore, learn.

Conclusio!

Like all of you, I could A) go on and on about all of the above, and B) provide even more things I’d like to see in CF 11 (like LiveCycle’s Reader Extensions for PDF!; or Spring integration, but Kotek shall continue to be the official bearer of that torch).

Know this: it’s OK to emulate! When I was a teacher, we had a saying: “The best teachers beg, borrow, and steal material to help their students learn”. Am I suggesting that CF become a copycat? Of course not. I will however state the obvious: CF has long since lost the “rapid application development” claim to other technologies. If it wants to reclaim it, it must adopt the features / tools / approaches that have caused so many developers to fall in love with platforms like Rails and Grails.

The wish list above includes the things that are most important to me and which I think would provide the greatest value – and Joy -- for the majority of ColdFusion developers.

Thanks for listening.

Make. It. Happen.

--Marc

Slides, Code, and some post-presentation question/answer are now available here: http://wiki.mxunit.org/display/default/CFObjective+2012+--+Concurrency+Zen

Thanks to all for coming out! I appreciate the enthusiastic audience with fantastic questions.

--Marc

I spent a few hours getting to know Grails. This post is from the perspective of someone not so much learning Grails as evaluating Grails. It’s not so much “Let’s have dinner and plan our wedding” as “Let’s have coffee and see if we wanna do this again”.

This is written against Grails 2.0.3.

What’s Awesome

• The Documentation. Well organized, well-written, well-formatted, and lots of code.
• The Shell. Once Grails is on your path, you type “grails”, and you’re in the shell. Apparently Grails used to have this thing called “interactive mode”, and the shell replaces that. Anyway, once you’re in the shell, you run your app with run-app, and *most* of your changes get picked up without a container restart. Some of us call this “F5” development. This is what I want
• The Console. On the shell, you can pop up a “console”, which is a simple text editor where you type in commands, execute them, and see results. This made it super fast to play with GORM once I created a few domain classes. Bonus points to all the keyboard shortcuts, even in the console.
• The Plugins. Egads. I typed “list-plugins” from the shell, and the list was so long I had to change my console buffer and re-run the command. I was looking for a few specific solutions: Flex, Excel, Mongo. All there, sometimes more than once.

Time to Un-WTF

When evaluating anything new, it’s typically only minutes before you hit your first WTF? That’s expected, and the absence of early WTFs would concern me. The question becomes “How long does it take me to go from WTF to not WTF?”. And, “What’s the process of getting there?”. For me, if I’m in a debugger trying to figure things out, that’s the worst. If I’m Googling, bad. If I’m on StackOverflow, bad. I want the early WTFs answered either in the app/code itself (via helpful error messages perhaps) or on the “official” docs. Let’s look at a few:

A New User

In the shell, I typed "generate-all helloworld.User" to generate a domain class, views, and a controller for a "User" object. My goal was to see the list of users, create a user, and edit a user. This is my story

Invalid syntax at user

After creating the scaffold files, I restarted my app. Bam, first error. Fortunately, it was kind of obvious, because Grails spit out exactly the SQL it was trying to run but could not. It was trying to generate the tables in SQL Server for my new User class, with sql something like: create table user (id....). I copied out the code, pasted into SS Mgmt Studio, and tried it myself. Same error. This is a good thing. No magic, easy to reproduce. And, for my simple case, the error is kinda obvious once it's in SSMS: "user" is a reserved word and I can't create a table with that name

Thus, for my first "time to un-WTF", I had to answer the question: "How do I specify a different table name for this class?". Impressive. First, I was able to find it in the official docs, without need to resort to Google. Second, because of how well the docs are organized, it was very simple to find. The docs don't have their own "search" feature, but I knew enough to go to the GORM section, and from there just pop up CTRL-F and search for "table name". I'm talking sub-minute time to Un-WTF here.

I added a row.... why won't it show up?

Next step was to create a new user. I opened the link to my newly generated controller, and it showed me an empty user list and a button to create a new one. Ok, let's press that. The form was empty, because I hadn't specified any properties on my domain class. Whatever... let's click that "create" button and see what it does. BAM. Error. I'll get to that later, because it was the most painful part of my first date. OK, so, creating a new record errors... Let's add one manuallly to the database, pop up the user list and see what happens

Nothing. No new rows appeared on screen, even though I could see them in the database. I mean, I know I don't have any properties on the class, but shouldn't the list of users at least have a "1" and a "2" indicating row number or something? Shouldn't there at least be some visual indicator that *something*, anything, is working, that it's actually pulling my data from the DB? So, my first guess is that something is caching results, and that since I added the data manually GORM doesn't know about it. So, off I traipse through the code to find anything that looks cache-like. I found 2, and I disabled them. Still, nothing. Grumble.

On a whim, I added a few properties to the class, and typed generate-views again. Hey, look, users! This obviously makes a lot of sense… if my newly generated domain class didn’t have any attributes, how would it be able to generate a meaningful scaffold for the “list” view? Of course it won’t show anything.

Here’s my (admittedly small) complaint: For anyone evaluating, they’re following the “Getting Started” guide, mostly. So they’re following along, typing “generate-all …”. Then they go to their app, hit refresh, and become wide-eyed with wonder and unicorns. ZOMG it’s all just there! Except, it’s not. Because even if you have data in your database, you see nothing. I have to imagine that others go through this. Bottom line: it’d be nice if there were  a simple visual indicator (a row number) that clues me in to what the real problem is. Maybe I’m an odd case in that I’ve been working with ORM long enough so that when something unexpected happens, I blame it rather than the more obvious culprit of simply having no properties.

The Magical _form

I admit to some surprise at seeing both a “create” and an “edit” gsp for editing a user. Where I come from, we keep this as a single template and use a little bit of logic to determine whether the user is new or existing, and adapt accordingly. It minimizes duplication. So seeing two files gave me the whillies. 40 lines of nearly identical code… Shudder. Fortunately, I  also saw this “_form.gsp” template in there, and upon inspecting the code I see this:

<g:render template=”form”/>.

OK, so at least the actual form isn’t being duplicated… just the bits around it. This leads me to my next WTF: why is the file named _form.gsp, but the g:render asks for “form”.

Since I was in the IDE, my first thought was to look at the grails.tld file to find some answers. No dice. So, back to the docs. I look in the right hand column of the page, and  I see Tags. Click that, it pops open, and I see “render”. Click that, and the page is mercifully short. A few scrolls down, and “_” is very obviously pointed out and described.

So, from my mullet-headed tiptoe through grails.tld to answering this question, sub 5 minutes. Not bad!

Hibernate Session Error

Anyone who’s worked with Hibernate enough comes to simply expect Hibernate session problems. Picture the movie Pulp Fiction, and the famous basement scene. Remember The Gimp? Yeah. That’s you. The Hibernate Session object is Zed, strapping the gimp ball around your face.

Anyhoo, here’s what I get upon my first attempt to save a new, *empty* domain object:

org.hibernate.HibernateException: No Hibernate Session bound to thread, and configuration does not allow creation of non-transactional one here

In my first hours working with Grails, I spent the most time on this problem. I’m not going to belabor it. I googled, I read all manner of docs, I swore. I couldn’t find anything. Finally, throwing out a hail Mary, in Config.groovy I changed grails.gorm.autoFlush from false to true. And then, just like that, it started working.

I have no idea why. I know I don’t like it. I know that sooner or later, I’ll need to figure this out.

What’s also troubling is that now, if I turn it back to false, I can’t replicate the error. So… was it because I had no properties? Was it something else I changed? And why didn’t google turn up anything obvious? Am I the only bonehead who follows (mostly) the docs, creates a new empty thing with generate-all, and then actually tries to run it? Probably.

I have zero doubt that I am the problem here. I’m used to it. I am unsettled at it nonetheless.

Questionable Practices

Code Duplication in the scaffold files

The duplicate files for create/edit puzzled me. That’s all… nothing more. I guess it sticks in my crawl because, as an evaluator, when working with codegen stuff, you base your decisions in part on the code that is generated. I, at least, expect the generated code in a Convention-over-Configuration framework to give me an idea of its conventions. It should attest to what it values. The generated code says “We, the makers of this framework, consider this to be a thing you’d want to do, too, otherwise we would not generate this code”.

Fat Controllers, No Services

Running generate-all, you get no Service objects. All the “work” gets done in the Controller. This is a very Bad Thing™, because the first thing that most of us are going to do is yank that crap out of the Controller, create a service class, and put it in there. Why this is not the default for the generated code mystifies me. I’d like to give these fine folks the benefit of the doubt, and perhaps all the smart kids know that “duh, you never use generate-all”.

A large part of learning a new thing is learning its community’s conventions. That takes time. A large part of evaluating a new thing is deciding whether what you see warrants that learning time investment. Just sayin’.

Conclusion and Next Steps

Even as I hit problems during my first date with Grails, I appreciated that it usually took me little time to solve them. This is huge for me when evaluating something… I expect problems. How quickly, and what steps I must take, to solve them are most important, and Grails performed admirably here.

That, coupled with the awesomeness mentioned above, most assuredly warrants a second date with Grails. In fact, several times during my few hours I felt like the guy in the big white tent, “Alleluia”-ing and “Testify!”-ing.

Next up I want to learn:

• How to expose a remote api, via json and amf. Is it something special, some annotation? Or is there “nothing to see here”? We shall see
• What steps are involved in hooking up a service layer? Given that there’s a ‘create-service’ command, and the project explorer in SpringSource Tool Suite shows a special marker for “services”, I presume this will be a simple as using and generating controllers. The only real question then will be figuring how how to inject dependencies, and I assume this is as simple as adding properties and letting Spring auto-wire them

I’ve got a question over on StackOverflow to do a bit of crowd-sourcing related to CFThread and Concurrency in ColdFusion applications. Please add your thoughts!

--Marc