#echonest

Musique: cela va faire du bruit!

MUSIQUE: Soundcharts annonce une levée de fonds de 2.65 M€ L’abonnement aux services de streaming est devenu le nouveau paradigme de l’industrie musicale, paradigme sur lequel repose désormais le marché mondial de la musique enregistrée. Il y a quelques jours, Goldman Sachs prévoyait que les revenus issus du streaming augmenteraient de 500% d’ici 2030 et valorisait dans le même temps Universal…

마이크로 음악을 들려주면 어떤 음악인지 알려주는 서비스인 Shazam의 작동 원리. 아이폰이 처음 나왔을 때 Shazam이나 SoundHound를 써 보고 마냥 신기해하곤 했는데, 이 글의 내용을 완전히 이해할 수는 없지만 대강 어떤 방식으로 작동되는지에 대한 감을 잡는데는 충분한 것 같다.

How do your currently search the web? You enter text-based keywords and phrases into Google, Bing or Yahoo, and the results you receive are based upon something (article, blog post, website) that someone has physically typed. These search results have to be read just as you would a book.

But what about the things that people say or do? How do you find those valuable pieces of information? I don’t mean when someone writes an article about something President Obama discussed during a State Of The Union address, or Kanye West’s latest rant in an interview. Don’t get me wrong, am a Kanye fan, and think he’s a creative genius. But I’m talking about searching the actual video file of these moments. Or the audio file of the song “Me Against The World”, when Tupac says, “Even the genius asks questions”.

Some have touched on the subject.

YouTube Transcripts: YouTube offers written transcripts of many of its videos. This helps them to be searched, as well as referenced as needed.

Rap Genius Annotations: Rap Genius is an amazing website. They annotate rap music, poetry, speeches etc. And many times directly from the author themselves.

EchoNest’s music intelligence application provides data that drives the information your receive about music from sites like rdio, vevo, and spotify.

Why would you build this? What is the benefit?

1. A more robust user experience for searching information. There are videos and audio on the web that currently cannot be searched past their written title.

2. The web is driven by video. People no-longer want to sit and read tons of text, although I hope you are reading this post in its entirety. We have become a world of instant gratification. People are busy and focuses on other things. Maybe the Internet is causing what I would call Voluntary ADHD. None the less, people like to see imagery. It’s what gets them to stop and pay attention in a busy world crowded with distractions.

3. Marketing and PR: When an advertising agency or PR firm wants to do reputation management, they want to know what is being said, by whom, and where, at all times when it comes to their client. It’s easy to Search the web for your client’s name or a topic keyword. Or to search social media #hashtags, set Google Alerts or even scour Google Blogs. But again, all the results you will receive will be text-based. You will miss when something is mentioned in a speech, song, or other audio or video clip.

4. Brands: How do brands currently figure out who to sponsor? Wouldn’t it be nice to know that rapper J. Cole regularly references Jordan Brand sneakers in his music? And of course he does. He is from Fayetteville, NC, and grew up playing basketball. In a situation like this, a brand like Jordan Brand could use HearSay Search Engine to listen to music for more than the music itself. This can be a powerful learning tool to understand who best matches your brand and represents it in a great light already.

In he future, the search results you receive should be able to be Heard and Seen. Information needs to catch up with the times and needs that the public has set.

But can data and information about the key elements of successful songs help artists craft the ultimate hit?  Some might guffaw at the idea, but Chief Data Architect at Next Big Sound, Eric Czech, found that by crossing the properties of tracks with sales numbers, he was able to identify certain factors that are instrumental to a successful song. In other words, there is a reason why Bruno Mars' "Locked Out Of Heaven," was more successful than Robin Thicke's "Brand New Luv."

Taking advantage of acoustic attribute data from Echonest (a subsidiary of Spotify) on the properties of tracks, ranging from tempo and duration, to acousticness and danceability, and crossing this with sales data from iTunes, Czech was able to analyze the relationship between any of these given properties and sales. Initial research shows that no one feature stands out as highly indicative of success, particularly given that the data set included tracks from a variety of genres, but that there are certain ranges for several of the properties, over which sales peak.

In addition, the research shows that the relationship between these features and sales is not linear, meaning that an element may have a positive correlation with sales, but only up to a certain point. For instance, increased danceability of a track – a measure of the intensity and activity levels of a song – helps it sell. However, if the level of danceability gets too high, sales tend to taper off.

This means that any predictive or normative conclusions that could be drawn from this analysis are the result of sales being maximized when several of the acoustic attributes fall within an ideal range. Simply put: tracks that are between 3 to 4.5 minutes long sell more. Tracks that are louder on average are more likely to catch the attention of your audience, to a point (between -9 and -1 dB). And while happier, acoustic songs can fare well, those that incorporate electronic instruments, and have a more mellow vibe, are prone to better sales, such as Passion Pit's "Take A Walk," or Calvin Harris' "Sweet Nothing."

In the past year, only about 345 artists, with 485 different tracks hit the Billboard Hot 100. Given the hundreds of thousands of artists in the Next Big Sound database, this is a fairly small proportion. Around labor day each year, the press scrambles to determine what can be deemed the "Song of Summer," now often based on months of prolific play, both online and on radio. In the running for this summer's title was everything from Iggy Azalea's "Fancy," to Megan Trainor's "All About That Bass." While they may not have been tailored for success in a data-driven manner, each of these tracks have a balance of key properties that helped them resonate strongly enough with audiences that they attracted millions and millions of plays.

While no one is arguing that one should take the art out of song-writing,  it is clear the data can help identify what those key properties might be. And in the competitive landscape of the music industry, a little help can never hurt .

Photo Credit: Wikipedia

A few friends and I spent the weekend at a Microsoft-sponsored USC vs. UCLA hackathon. We worked diligently for 24 hours and made a pretty sick app that I’d like to share.

Also worth noting, our app actually ended up winning the overall hackathon. We barely scraped past the second-place team, who had a really killer idea as well. Either way, I’m super proud of my team and what we were able to accomplish, and I’m excited to share a bit of the experience with you.

The Team

For this hackathon, I teamed up with a couple of friends from USC, and our mission was to make ‘something dope’. We called ourselves SPDerp after our fraternity letters, Sigma Phi Delta. We sorta divided into front-end and backend, with Jesse Chand and Albert Kuo working on the front-end of the app while Daniel Silva and I crafted the backend.

Also a quick shoutout to Kabir Gupta who joined us for the Hackathon but due to team-size-restrictions couldn't be a part of SPDerp. He worked on his own dope application which you can check out here.

Being that the hackathon was sponsored by Microsoft, we were developing for either the Windows 8 or Windows Phone platforms. Since none of us had any experience in either of these platforms, we decided to make a web application instead. So our entire app is made up of native javascript and HTML, with a handful of calls to a couple of PHP scripts in cases where API calls could not be made through javascript due to cross-site-scripting security restrictions.

The Mission

So this hackathon had a theme : Mash-ups. Essentially we were asked to take two different APIs and mash them up together to create something interesting. It was a tough challenge since we’d have to learn how to use the API, how to get the info we need, and combine the APIs in an interesting way.

The Process

We started with Flickr. The truth is that I love Flickr. Where else can you get links to gorgeous, professional-quality images, with a brilliant indexing and querying system? But the real reason that I love Flickr is because it makes it easy to make an app look sexy. If you don’t believe me, take a look at the Yahoo! Weather app, which is almost certainly the most beautiful weather app that has ever existed.

Anywho, we decided that Flickr would be a good place to start, and so we started thinking about what we could do with a bunch of easily-queryable, high-quality images.

And, we thought, images can portray mood as well or better than words. Even an image of a landscape can resonate with a person on an emotional level. So perhaps we could analyze the mood of an image, get some basic mood data, and use that to do something interesting.

That something interesting, we decided, was to create a music playlist.

The App

We created Moodsic: a web app designed to create a perfect playlist by using analyzed image data to determine the person’s mood.

And since it’s all built with native javascript, I’ve hosted the app so that you can try it out for yourself. Quick disclaimer: We weren’t able to spend as much time as we’d have liked to on our image-mood-song analysis algorithm, so the generated playlists are somewhat arbitrary. But the logic is all there and it just needs to be fine-tuned to produce solid results (which we, of course, plan to do).

Feel free to give it a whirl– you will need to have Spotify open (either the native or web player) in order to actually play the generated playlist, but you get the gist.

The Process

If you’re not interested in any of the technical mumbo-jumbo/nitty-gritty/other-cliché, then you might want to skip this section.

Basically, we started by creating a native HTML/css/javascript environment. We designed it to look solid on a tablet (since it was to be demoed on a W8 tablet). In that light, it probably doesn’t look or feel too great on mobile.

The Images

So the Flickr API was easy enough. We decided to use a filter to only select images of landscapes, seeing as it’s easier to decipher the mood if there aren’t any actual people in the image. We were then able to restrict ourselves to very simple heuristics like average red, green, blue values, and brightness of the image.

We were able to extract these heuristics from the images by using the British Columbia Genome Sciences Center’s Image Summarizer API. We were originally set to use Imagga, but we weren’t quite able to get it to work and had to abandon it mid-hackathon.

Once we got the average red, green, and blue values for the image, as well as the images brightness, we were able to determine the ‘mood’ of the image. We got this by essentially mapping all of the possible combinations of different colors to some base emotion:

Low red, low green, low blue : dramatic

Low red, low green, high blue : calming

Low red, high green, low blue : epic

Low red, high green, high blue : reflective

High red, low green, low blue : angry

High red, low green, high blue : rebellious

High red, high green, low blue : aggressive

High red, high green, high blue : light

Admittedly, for this app to really work as desired, we’d have to incorporate some kind of machine learning algorithm as well. But for a 24-hour hackathon, this did just fine.

Also worth mentioning : based on the brightness of the color, we determined the minimum and maximum tempo (BPM) to search for. A bright image would return faster, higher BPM songs, while a darker image would return slower, low BPM songs.

The Songs

Once we had this image analysis on the three images that the user selected, we turned to the Echonest Song Search API to grab twenty random songs based on this information.

Finally, once we have a list of songs, we add each to a spotify playlist and display the music player.

An Interesting Challenge

I’d like to take a moment to explain one of the hackiest parts of our app: the php xml-jsonp wrapper that I built to allow us to query these XML based APIs from native-run javascript.

The challenge is that javascript queries are generally reserved for domain-level queries. Essentially, the javascript running on facebook.com can access any data or resource that is hosted on facebook.com. But if facebook needs to grab some Twitter data, for instance, it cannot do that through the client-side javascript, because this results in some crazy security issues.

So if I want to grab the 20 most dramatic songs from echonest via their api, I’d need some way other than AJAX to grab that information.

Enter jsonp, AKA json with padding. What I was able to do was use a very small PHP script (hosted on my site), to forward an incoming query to the Echonest API. Then, I’d wrap the resultant XML in a javascript method call, and then send it to the native app. This is allowed because the information must be in JSON-acceptable format, which restricts most cross-site-scripting vulnerabilities.

The Takeaway

This app was awesome to build. We wanted to make something really interesting that did some crazy magic behind the scenes to deliver awesome content, and an awesome experience to the user. We wanted to make it super enjoyable to use, and eventually we’d like to to learn how to make better predictions.

Either way, I think I speak for my team when I say that we are all humbled to have won the competition (and a free trip up to Seattle to visit Microsoft’s HQ). I personally had an awesome time, and I’m especially proud of how much we got done in the limited timespan of the event.