Simple Geospatial Queries with MongoDB

Tuesday, November 23, 2010

MongoDB has supported geospatial queries for a while, and in the upcoming 1.7 release it’ll get even better. Let’s take a look at how easy it is to query MongoDB in an idiomatic geospatial manner.

First, as described in the docs, you’ll need your data structured a certain way, and then you’ll need to add a 2d index. In my example, I’ll have documents that contain non-geospatial data (names, dates, other stuff) and then an embedded “LOC” document which contains “LAT” and “LONG” fields. These are arbitrary names… what’s important is that you get the order right. First, your “location” field will contain 2 values. You can do this in two ways:

  1. array
  2. nested document with keys

For example:

{  LOC: [38, –102] } is a valid “location” field because it contains an array with 2 values

{ LOC: { LAT: 38, LONG: -102 } } is a valid location field because it contains a nested document with two keys, and those keys contain numeric values. 

Note that the order is important! You’ll insert the data in the order in which you’d query the data. In general, stick with lat/long or x/y order.

Sample data

geodata

As you can see, this “treatment unit” is positioned at roughly [38,-102]

Adding the 2d Index

Let’s add an index:

> db.treatmentunits.ensureIndex( { LOC: “2d” } )

Cool, that was easy.

Querying with geospatial operators

Exact matches are rarely useful unless, and become even more so the more granular your lat/long storage becomes. You’ll notice in this example that I have ridiculously precise location data (it’s precise b/c it’s fake). Still, if you want to query for exact matches, you’d do this:

> db.treatmentunits.find( { LOC: [38,-102] } )

Due to the precision of my data, this will yield no results. So let’s widen the net by using the “$near” operator:

> db.treatmentunits.find( { LOC: { $near: [38,-102] } } )

That’s better… lots of results. By default, Mongo will give you the closest 100 results. You probably don’t want that. So, let’s tighten it up by setting some “bounds”, using the $maxDistance operator:

> db.treatmentunits.find( { LOC: { $near: [38,-102], $maxDistance: 5 } } )

You might be thinking: how’s that different from using limit(x)? Simple: limit() is merely a limit on the number of rows returned. If you want only 5, you get only 5. But by using $maxDistance, you’re not specifying how many results you want but rather how close the locations must be to your target location in order to be included in the results. If you want the closest 10 locations that meet a $maxDistance of 5, that’s where you’d use limit:

> db.treatmentunits.find( { LOC: { $near: [38,-102], $maxDistance: 5 } } ).limit(10)

Now, picture a map in your head, and draw a box somewhere on that map. Let’s say you want to find results whose location is within that bounding box. You’ll use the $within and $box operators, like so:

> db.treatmentunits.find( { LOC: { $within : { $box : [[40,-120], [48,-108]] } } }  )

Now that’s a big box, comprising nearly the entire southwestern US, so it’s going to return a lot of records. Tighten up the box to limit your results.

Finally, to find records whose location starts at a center point and radiates out, you’ll use the $within and $center operators:

> center = [38, -102]
> radius = 10
> db.treatmentunits.find({ LOC: { $within: { $center: [center, radius] } } } )

You may be wondering at this point, “Why don’t you have any sort() applied?”. The answer is: MongoDB will provide the correct sort for you when using geospatial queries

Debugging Flex/AIR Applications on an Android Device


At RIAUnleashed 2010, I attended a lab titled "Building your first Android application using Flex and AIR". One of the first things you'll want to do is see your "Hello World" app running on a device, and not just the emulator. On Mac and Linux, you shouldn't need to do anything special. On Windows, however, you'll need to do some work.

Note: This assumes you have Flash Builder Burrito installed. If not, you can get it at labs.adobe.com. In addition, I’m using a Droid 2, which is a supported device.

Note 2: You need wifi. Though you’ll be using USB to deploy the app to the device, the actual “debugging” happens over wireless. If you don’t have wifi, you can still run the app on the device, but you won’t be able to set breakpoints in your code, etc.

First, if you're new to Android, and you haven't yet attempted to connect your device to your PC yet, there's one small but important thing you'll need to do, and it appears that it does not do this by default, leading to potentially a lot of time and frustration.

I had two different installation experiences on two different machines. On a Windows 7 AMD64bit laptop, installing the Android drivers for the Droid 2 worked exactly as described in the Adobe docs. However, on a Windows 7 IBM 64bit PC, I had no luck installing the drivers manually. To finally get the Droid 2 installed properly, I did the following:

  1. Connect via usb (duh)
  2. slide the notification panel down and select "USB Connection".
  3. It will give you several options, and probably defaults to "Charge Only". Change this to "PC Mode", and then it'll install the appropriate drivers from the web. If you connect the device with “Charge Only”, you’ll bang your head against the wall trying to figure out why Flash Builder can’t connect.

This should be all you need to get FB to recognize your device. Now, you’ll need to change some settings on the device to “open it up”, namely enabling USB debugging.

  1. On your droid, go to Settings
  2. Choose Applications –> Development
  3. Enable USB Debugging

Now, back in FlashBuilder, do the following. This assumes you have a Mobile project:

  1. Right click either in the editor of your “main” file or in the package explorer
  2. Select “Debug As”, and then “Debug Configurations”. You can also choose “Run As” to run it, though it won’t be in debug mode.
  3. Select “Mobile Application”, and then  click the “New” icon.
  4. Give it a name, like “My App on Droid 2”
  5. In “launch method”, choose “on device”, and then hit “run”
  6. This should pop up a grid with connected devices, and your device should be in there.

Note: on the PC, it took about a minute after installing the drivers before FlashBuilder would recognize it as a connected device. In fact, I changed it from Charging to PC mode several times, and then finally it showed up in the grid.

Finally, after getting the drivers installed (if you’re on Windows), correctly setting the Droid to enable USB debugging, and configuring a FlashBuilder run configuration, you should be all set for debugging on your device. If you still can’t debug, it might be due to being on a VPN, or also due to firewall settings. You’ll need to correctly configure your firewall and perhaps your wireless router to enable the communication between your dev machine and your Android device.

Imagine If you could design your own conference…

Sunday, November 21, 2010

OK, so designing your own conference is probably not realistic. But with cf.Objective() 2011, you can at least let your voice be heard. Go to http://engage.cfobjective.com and submit topics you’d like to see. Vote on previously submitted topics if you see topics that speak to you. Comment on those topics if you have additional input.

You’ll note that we’ve provided for ‘suggested speakers’, but we’re not showing those names. Why? Simple: this isn’t a speaker popularity contest. We have a lot of great ideas for the kinds of sessions we want to include, and we need your input to both confirm what we have in mind and also to add content we haven’t thought of. Please… submit and vote!

It may also be helpful to think in terms of “What’s the single topic I absolutely, positively want to see?”. Add that as a comment, please. We’ll be looking not just at raw number of votes but also at other trends that emerge in the comments.

If you want to know more about why we’re doing this, Jared explains it all.

http://engage.cfobjective.com

MXUnit 2.0.2 Released

Thursday, November 18, 2010

MXUnit 2.0.2 is now released! This small update provides:

  • a bug fix to JUnitXMLTestResult
  • some HTMLResult tweaks
  • a new restoreMethod() to complement injectMethod()

Release notes are here, along with the download link.

Thanks to everyone who contributed code for this release!

--marc

MXUnit Eclipse Plugin 1.4 Released

Wednesday, November 17, 2010

It’s been way too long since I’ve written about an update to the Eclipse plugin. Quietly, some time back, I added a few niceties but never announced it. Recently, Denny Valliant of CFEclipse renown sent me a patch with several feature additions, and this prompted me to finish work on something long overdue in the plugin.

The new hotness:

  1. RemoteFacade URL at any resource level… not just project. This is particularly useful if you separate your tests into “unit”, “integration”, and perhaps even “ui”. Now, you can stuff relevant functionality – ORM Reloading, Selenium server startup, etc – into a custom remote facade for a given folder of tests. Prior to this addition, custom RemoteFacades usually had gnarly logic to determine what type of test was being run and what action should be taken as a result. No more
  2. In addition, when you set a custom remote facade, you can ping it from the Property page and get an HTML response back to confirm all is well
  3. When you run a test in the plugin and something catastrophic happens – something the plugin can’t handle – you’ll now get a popup browser window showing you an HTML version of the error. This should help considerably in debugging communication problems between Eclipse and ColdFusion
  4. The plugin no longer “hangs” Eclipse when it starts its initial communication with ColdFusion. Previously, if CF was shut down, or if it had just start up, attempting to run a test from the plugin would often hang the entire IDE while it waited for ColdFusion. No more.

For first-time installers: The update site url is http://mxunit.org/update

As always, if you hit any problems, let me know.

And, as always, if you have any problems figuring out how to configure the plugin, PLEASE click the green help icon in the plugin! In addition, see this page and its children on the MXUnit wiki for troubleshooting and other tips and tricks.

Finally, a huge “Thanks!” to Denny. You’re a code machine, bro.

ColdFusion Skunkworks Slides and Materials from MAX 2010

Tuesday, November 16, 2010

At Adobe MAX 2010, Joe Rinehart and I presented a session titled “ColdFusion Skunkworks”. What an experience!

I’ve posted the slides and materials on the MXUnit Wiki, and a recording of the presentation is available on Adobe TV. Please check it out. As I wrote on the MXUnit wiki, this presentation is our ode to playful programming, to putting aside, if only temporarily, familiar approaches, to taking risks, and to exploration.

I’d like to say “I hope you take as much from it as I did,” but I do not believe that is possible. This experience was transformative for me, to be sure. I’ve given a number of presentations at conferences, but none of them led me to learn so many new things as this one. Joe and I decided that if were were going to present on a topic as bold as skunkworks, then we needed to live it ourselves. It was fun, exhausting, exhilarating. So how about this: I hope you get a glimpse of what can happen when you devote some time to you instead of giving it all to the man. Go play!

--marc

MongoDB Slides and Materials from RIAUnleashed 2010


Picture, if you will, a ColdFusion Structure. Perhaps it looks like this:

doc =
        {
                NAME = "Marc",
                SPOUSE = "Heather",
                KIDS = [
                        {NAME="Alexis", AGE=7, HAIR="blonde", DESCRIPTION="crazy" },
                        {NAME="Sidney", AGE=2, HAIR="dirty blonde", DESCRIPTION="ornery" }
                ],
                BIKE = "Felt",
                LOVESSQL = true,
                LOVESMONGO = true,
                TS = now(),
                COUNTER = 1
        };

Now, imagine if you could save that struct, query for that struct, update that struct. Imagine if you had a whole database full of structs, and querying them was finger-snap fast. Imagine if the database they lived in could grow to sizes unmanageable in a relational database. Imagine if scalability were sane, and replication were a few commands.

This database exists today, and it’s called MongoDB. MongoDB is a Document-oriented database, like CouchDB. A major player in the so-called “NoSQL” database market, MongoDB is a pleasure to use, partly because it aims to bridge the gap between NoSQL and relational databases by providing ad-hoc query support similar to SQL while still giving you MapReduce which most Document-oriented datastores use as their main query mechanism.

I began working with MongoDB when I began at BAH in May, 2010. I was tasked with creating a system whose data model was rather heterogeneous. We modeled the system first in a relational database, and while the result was familiar, it was also the embodiment of the “Object-relational impedance mismatch” which developers have bemoaned since the onset of Object-relational mapping systems. Ideally, the heterogeneous data could be persisted directly, without splitting apart before storing and reassembling upon fetching.

With MongoDB, we were able to achieve such data persistence. Thanks to that experience, I continued to work with MongoDB, and this November I presented on the topic at RIAUnleashed in Boston. This was also the first public presentation demonstrating CFMongoDB, of which I and the rest of the team are becoming quite proud.

The slides and sample code are available here: http://bit.ly/mongodbriau. Enjoy!