Took a left turn with the SimPuebo project, kind of. I realized that the polygon matching I was trying to do with the outlines of the buildings was going to fall apart.  I had originally thought that I’d still be able to split them apart when they moved next to each other, but that turned out to be a can of worms I didn’t feel like opening.  Next try is putting triangles made out of green dots on the buildings and identifying them that way.

Looks like the algorithm does a pretty good job correctly guessing buildings, but oh man does it go nuts when the buildings touch!  Also my filters might be too sensitive to lighting conditions.

Finally! An answer to my issues with the Sprout recognizing individual buildings.  I think I’m going to use the Turning Function method.  This will probably also help determining if two buildings are touching, and if so, which two and at what angle.  Yay!

Do I seriously need two large image libraries in order to process output from the webcam without previewing it? The first webcam library I used cheated by copying the image from the screen in order to save it, not directly saving the frame from the webcam.  Why is this all one big pile of easier-said-than-done?  :-(

Needed this to get the info from the downward facing camera on the Sprout

Decided the best library to try for image feature recognition would be OpenCV.  Then I would use the Maximally Stable Extremal Regions algorithm and compare the regions to the buildings.  Hopefully. There’s a C# wrapper for OpenCV, but it appears to be a nightmare to install unless you use the shortcut.  Man I hope this works.