#mealsnap

This is beautiful! Read this, it's jawdropping.

So, there's an app for iPhone called Meal Snap where you take a photo of your meal and the app "magically" works out what the food is and how many calories are in it. Amazing, huh?

Maybe not. I got to thinking about how hard it would actually be to write software to do that, and I concluded that the easiest way would be to send the photos to a human and ask them to identify the food and look up the calorie content. But could Meal Snap really have an army of people sitting in banks and labelling photos of food?

Kinda, yes. I didn't know about a fiendish Amazon intiative called Mechanical Turk which pays people piecerate to work on what they call Human Intelligence Tasks, for example looking at a photo of a meal and identifying what food is in it. Blogger Justinxreese from I've Noticed spotted that Meal Snap are using Mechanical Turk to pay people 2-5 cents a picture to do just that.

How cheeky is that? But I have to say, I love the chutzpah!

I was recently talking with someone about a web project of mine that has, at its core, users tagging meals they ate with the foods contained in those meals. I was mentioning that I spend a great deal of time bringing the friction point of tagging those foods down -- making it as easy as possible. They brought up MealSnap, an iOS app that has users take pictures of food, and then magically identifies not only what foods are contained in that picture (magic!) but also the caloric content of that food (doppel-magic!).

I couldn't stop thinking about how they do it. Do they start with the color of a food, and then do some sort of edge detection? Do they have a massive database of images, and it somehow searches for similar ones? Turns out it's simpler: they upload the pictures to the web, and then pay actual, real humans to type in what's inside:

Looks like they’re using Mechanical Turk to identify the foods at anywhere from $0.02-$0.05 per picture and then using the data returned from Mechanical Turk to search for calorie information in their already well established database of food.

With a $2.99 price point for the app, DailyBurn would start losing money at around meal 60. By that point, however, Daily Burn has a loyal user that can easily be converted to the sale of another app in their family of products.

(Thanks, justinxreese). Kind of an ingenious shortcut to the simplest solution to a problem: a human can easily tell what's chicken and what's fish. A computer program to do the same might take gazillions of dollars to implement well. It's easier to pay a modest fee to a well qualified human, e.g. one with eyes and a brain and a reasonable command of English.

This Mechanical Turk business? It's an Amazon Web Service. Why turk?

The name Mechanical Turk comes from "The Turk," a chess-playing automaton of the 18th century, which was made by Wolfgang von Kempelen. It toured Europe beating the likes of Napoleon Bonaparte and Benjamin Franklin. It was later revealed that this "machine" was not an automaton at all, but was in fact a chess master hidden in a special compartment controlling its operations. Likewise, the Mechanical Turk web service allows humans to help the machines of today perform tasks they aren't suited for.

Thx, wikipedia. It turns out when MealSnap says their app is "Magic. Pure magic," they're not joking. What is magic but taking something that can't possibly work, is against the laws of the universe as we know it, and faking it in a believable way for an eager audience?

If you've used DailyBurn's new app, MealSnap, you've probably been mystified wondering how it works. MealSnap is an app that lets its user take a picture of the food they're eating and, based on that image, will provide calorie information for that food.

I've used app for about 6-7 meals now and I've been pretty amazed at the accuracy. So much so that I had to find out how they did it.

There has been some speculation at how it works in articles, but I couldn't find a solid answer. Is it image recognition? Do they have a team? Do they use Mechanical Turk? Well... one of those hypotheses is incredibly easy to test. 

I visited Mechanical Turk, typed in "food", and ta-da! 

Well.. truth be told, I had to hit refresh because these HIT's are moving fast and the first search turned up no results. Looks like they're using Mechanical Turk to identify the foods at anywhere from $0.02-$0.05 per picture and then using the data returned from Mechanical Turk to search for calorie information in their already well established database of food.

With a $2.99 price point for the app, DailyBurn would start losing money at around meal 60. By that point, however, Daily Burn has a loyal user that can easily be converted to the sale of another app in their family of products.