Hackerati | Coded & Crafted

Our depiction of the human face hasn't remained constant, however. Using a technique called face averaging, we can see how the art of the face has evolved.

For each of the images below, I've taken between thirty and ninety portrait paintings, depending on how many I could find, from a given time period. Then I averaged all of those together. So each image you see is not one person from one painting, but represents the average face painted in that period of art.

Renaissance 1300 - 1600

Baroque 1600 - 1730

Rococo 1720 - 1780

Neoclassicism 1750 - 1830

Romanticism 1790 - 1880

Impressionism 1860 - 1890

Post-Impressionism 1886 - 1905

Cubism 1907 - 1914

Only 18 out of 90 cubist images made it through the facial recognition software!

And here is how we portray the human face today. The following two images are each averaged from 100 different digital paintings from deviantart:

And here are some of the above images side-by-side, to more easily see the differences:

Of course, that which is truly beautiful about a face is that which is unique to it, along with the particular expression on the face in a given moment. By averaging hundreds of faces together to form one image, we've clouded the expressivity and uniqueness of the individual paintings. Even so, some expression still shines through. We can see the reservedness in some face average images, the sweet joy in others, and the seductive quality in yet others.

How will our depiction of the human face change over the next thousand years? I look forward to it!

Luckily, there are models that have been trained on different images! Using different datasets yields much more pleasant deepdreams.

Places

The MIT Places dataset was trained on images of various indoor and outdoor scenes.

When we've been looking at beautiful places all day long, that's what we dream about. These deepdreams tend to be dominated by pagodas, fountains, bowling alleys, stadiums, tree-lined circular paths, coliseums, and baskets of apples.

Cars

The GoogLeNet Cars dataset was trained on different types of cars, using outisde views of varying angles, closeups of headlights and taillights, and pictures of the driver controls.

When we dream, we see smooth swaths of metal, lots of curves, headlights and taillights everywhere, and the occasional wheel. These are the spaceships with which we will send our slugdogs into outerspace, far far away from us.

Age and Gender

The Age and Gender dataset was trained on faces.

We dream of creepy babies and old folk with lots of makeup. The arch of the eyebrow and the wrinkles around the eyes are really important.

Flowers

The Oxford Flowers dataset contains closeups of flowers such as the following.

This model is fine-tuned from the reference CaffeNet. Instead of training from scratch, we can use what we have learned before. However, this does mean that in certain layers the dogs will come back. But we will avoid those layers!

Flickr Style

The purpose of the Flickr Style model is to categorize photos into styles (such as serene, melancholy, bokeh, and noir) and into art genres (such as Early Renaissance, High Renaissance, Ukiyo-e, and Impressionism).

Like the Oxford Flowers model, this too is fine-tuned from a broader model. We dream in swirls and textures.

More Models!

You can play too! All the models above are from the Model Zoo.

Here’s the repo:

https://github.com/thehackerati/node-app-template

What’s Inside

Docker is the heart of this environment.

For local development on a Mac, we rely on Vagrant to provide a Linux virtual machine within which we can run the Docker images. If you use Linux for development, you can run docker-compose directly.

The development environment allows you to edit source in your favorite text editor or IDE in your development host, while the application automatically runs tests and reloads inside the Docker container.

Travis-CI automatically runs tests and deploys builds to staging and production environments when you push and merge code on Github. You can see the build running here:

https://travis-ci.org/thehackerati/node-app-template

Staging and production environments run Docker images on AWS Elastic Beanstalk. You can see the code running here:

http://nodeapptemplateprod.elasticbeanstalk.com/

http://nodeapptemplatestaging.elasticbeanstalk.com/

There are two permanent branches in this repo:

master: automatically deploys to the production environment on push or merge to master

staging: automatically deploys to the staging environment on push or merge to staging

Setting Up the Environment

To contribute, you can start by forking this repo and cloning your fork to your local development environment.

To use this template as the basis for an application, clone and rename it, and push to a new repo on Github.

Once you have a new repo on Github and a local clone on your development machine, you can setup the rest of the environment:

Create automated build on Dockerhub

Connect Tavis-CI to Github repo

Run scripts to setup staging and production environments on AWS Elastic Beanstalk

Run vagrant up

See the README for detailed instructions.

The second component seems to look at short dresses vs long dresses.

Reddish colors vs blueish colors.

Short hair and sleveless vs long hair and long sleeves.

Posing with legs close together vs posing with legs farther apart.

And so on.

Using components to recreate images

With a bunch of components like these, we can reduce an image from, eg 60,000 points of data (pixel values) to just a handful of numbers.

Let's recreate this dress from its components.

The following pictures are created from one, two, four, nine, ten, fifteen, thirty, forty, and seventy components (respectively).

As you can see, the more components we have, the more accurate and detailed the dress recreation will be.

The data for the middle dress above now looks like this: [-17541.81, -12749.33, -3766.29, 2005.28, 4193.08, 6832.55, -6704.90, -2135.51, 1112.27, 7627.80].

So, if you have a million pictures you need to store, you can save a whole lot of space by saving just the component values instead of the values of every pixel of every dress.

It even works for dresses that were not in the training set:

Though it works less well for patterns we haven't seen before:

And can't recreate accessories that were not present in the training set (notice the sunglasses and handbag disappear):

And even though the training set only contained dresses, the data is decent at recreating different types of clothing, such as suits and overalls:

Using components for prediction

I've also manually categorized the pictures as dresses I like (287 pictures) and ones I dislike (520 pictures).

Now we can use logistic regression on component data to predict whether or not I'll like a dress.

Sorting all the dresses by score, it can show the prettiest and ugliest dresses of the whole set.

The prettiest dresses:

The ugliest dresses:

Seems pretty spot on! I could now set up something to watch as new dresses are posted on Amazon, and to alert me to dresses it thinks I will really like.

The misclassifications are interesting too. Here are the three "ugliest pretty dresses", those that I classified as my style, that the program predicted I should really dislike:

It seems to be about that specific shade of blue.

And here are "prettiest ugly dresses", those that I classified as dislikes, that the program predicted I would really like:

These aren't that bad. I do kinda like them, but think they'd be nicer with some minor adjustments (slightly less form-fitting, slightly less loud pattern, slightly brighter color).

Creating new dresses

For creating pictures, there's no reason we need to confine ourselves to already known dress component values. We can also choose random values for each component, and see what happens!

Completely new dresses! With more data and better data, this could actually be a viable dress design tool!

Want to play? Code on Github

-- Grace Avery

If you'd like to see more about neat applications of PCA, check out Joel Grus's post!

Many engineering teams have adopted Git Flow, especially for release oriented projects like native mobile apps, where the cost of deployment is generally higher. For projects where continuous deployment is practical, others have adopted Github Flow. Whichever approach you choose, the key is to use a set of practices that your team can follow and that is as simple, lightweight, and automated as possible. 

Agile engineering relies heavily on automating common development and deployment tasks. Figuring out automation in 2015 can be a nightmare of choosing between a myriad of tools: Git, Vagrant, Boot2Docker, Machine, Make, Chef, Puppet, Ansible, Docker, Jasmine, Mocha, Protractor, Karma, Jenkins, Travis-CI, Shippable, AWS, Heroku, Rackspace... There is no shortage of great technology choices and sample code to show you how it works. Unfortunately, there are few working examples of how it all works together, end to end.

We’re building a series of open source application templates that implement a point of view of on agile engineering tools and practices, supporting the workflow described in the diagram below, end-to-end from local development environment to staging and production environments.

With these application templates, you will be able to setup local development, staging, and production environments in less than an hour, complete with automatic reloading and running of tests on save in the local environment and deployments on merging pull requests to staging and master branches, respectively.

Our goal for these NYC Tech Talks is to share the in-depth insights we’ve gained from engineering successful technologies across various industries. We expect to focus on topics including Web/Mobile, Big Data/Analytics/Visualization, Machine Learning, and Distributed [Cloud] Computing with presenters from The Hackerati and other leading NYC Tech companies.

We’ll be kicking things off with a presentation by Rob Marano, Cofounder of The Hackerati. He's planning to show us how to leverage new AWS components and Spark to analyze clickstream data for online advertising.

Your coworkers, your friends, even random people off the street will do the same. They'll lie through their teeth when giving feedback, positive or negative. Consider any of the following motivations:

1. You mean a lot to them, so whatever you give them they'll tell you it's awesome.

2. They hate you, and no matter what you give them they'll tell you it sucks.

3. They pick a small detail that they can critique, just so they can say enough words to not make the conversation awkward.

4. They actually use a totally different operating system, and lied about their habits so they can get paid.

5. Maybe they are telling you things as they are, but not how they should be.

[http://xkcd.com/688/]

You'll find a couple of awesome people who can give you the appropriate amount of tough love—like your boss, if you’re lucky. But how do you get a statistical representation of your user base for those tricky nuanced questions? What if your target audience is teenagers and not an assortment of designers, developers, and project managers?

Just like good design is invisible, good feedback is mostly unconscious. Put your app in the user's hands and watch what they tap. Ask them to find the latest content and look at which buttons they press. Maybe they totally ignore certain menus, or have to double-back in an awkward way because they got lost somewhere. Even worse, maybe they give up on finding something because it's not in an obvious location.

For a more concrete example, maybe you’re concerned about the loading speed on a certain piece of content. One solution might be to set up timers and run studies about how many milliseconds are too long. Or maybe ask the user if it felt responsive. Whether they would wait that long. If they would open the app weekly.

The better way would be to go up to a random stranger, ask them to explore your new app, and see if they press the cancel button or start to hand you back the phone before the content loads. Try to have the least attractive person in your office do it and see if they still stick around.

To quote the entirety of one of my favorite app store reviews, "Cool."

With luck on your side and about 30 iterations of the above, maybe you too can achieve such stellar results.

As a people company, The Hackerati relies heavily on social contracts and nontraditional approaches to foster an enjoyable, creative, innovative culture. Our culture has been focused on transparency, authenticity, and having a helluva lot of fun from Day One.

As Chief People Officer, Amanda will lead and continue to develop our unique culture, with a focus on diversity and innovation. She’s also working on some exciting initiatives to collaborate with other NYC Tech companies and the tech community in general.

The Hackerati attracts and retains top engineers by offering a work environment that doesn’t really feel like work. With a blossoming startup accelerator [Hackerati Labs] as our core side project, there’s plenty of room for personal and team growth. And at the absolute forefront of everything we do is the motto Be Fearless. We understand that innovation isn’t a product of following the crowd, and we do our best to promote and celebrate differences. Catering to all of our team’s diverse interests—from hacking to music to breakdancing to learning languages to poetry—is a core internal focus, and this is why having Amanda Keleher on board as Chief People Officer is a huge step forward for us.

Though early in her journey as a Hackerata, Amanda has already provided a powerful burst of perspective and energy. We all feel fortunate to work with such a force, and are sure to look back on this event and see it as an overwhelmingly positive turning point.

TL;DR

Become your favorite actor or your crush’s fantasy and send them a pic! Clone the repo or get the first Beta release and try it out for yourself!

openFrameworks

The app is built on a framework call openFrameworks. It’s a C++ framework designed for creative coding, so you can develop really awesome and visually intriguing apps, and with all of the addons available the possibilities are endless. The app is also based on our very own Grace Avery’s OS X app as featured in a previous blog post which achieves the same result, just on Macs.

The App

Building the app posed a couple of problems. The first and most apparent “problem” is that openFrameworks libraries are written in C++ and iOS apps are, primarily still, writted in Objective-C. Luckily, there’s a way to combat the issue via Objective-C++. Essentially, the app is written both in Objective-C and C++ with both syntaxes mixed in to a single file in some cases. There’s some meddling around in the Build Settings required to get the app to compile correctly at first since you’ll get a bunch of type reference errors unless you use C++99 for the language dialect and libstdc++ standard library as oppose to the libc++ standard library. Secondly, ARC, or Automatic Reference Counting, isn’t a thing in the app. Garbage collection is all handled manually just like in the good ol’ days, so the app is sorta primitive in that sense. And also pointers. Pointers everywhere! They aren’t so much problems as they are a nuisance.

How Does This All Tie Together?

So enough with the nitty gritty… how does the app work? You can look at the video as 2 layers. The first (bottom) layer is what the camera is capturing (a.k.a. your face) and the second layer (top) is the face you have chosen to become. If you just slap the image you want on top of the bottom layer, it’ll be like plastering a poster onto your face, so we need a way to “cut” the face out of that poster. If we break that process up, we first need to actually find a face. That’s all handled through a Face Detection library, ofxFaceTracker, and a Computer Vision library,ofxOpenCV. The image (whatever face you chose) is converted into a matrix via ofxOpenCV and that matrix is fed into an ofxFaceTrackerinstance. Once we get that tracker set up we can get the image points of the face. This will allow us to create the mesh that will be our cut out face. For those interested in the code, here it is:

src.clear(); // src is the raw image src.loadImage(face);         if(src.getWidth() > 0) {    Mat cvImage = toCv(src); // create the point matrix    srcTracker.update(cvImage); // update the face tracker with the new face    srcPoints = srcTracker.getImagePoints(); // get the face points }

The same thing is done for your face on the bottom layer. For video face detection, we use the ofxFaceTrackerThreaded library to conduct calculations asynchronously to better performance and not bog down the system. The reason for using the threaded face tracker to get the points of your face on camera is that those points are constantly recalculated and found again almost every new frame of the video (sorry, can’t help dropped frames), so performing these operations asynchronously is crucial.

What you’re left with is two sets of face points that can be converted into a mesh. Each set of points represents a face whether it be your face or the selected face. The points in each mesh can be mapped to each other 1-1, so essentially it’s like taking a rubber mask and stretching it to cover your face while keeping the major features as close together as possible (in this case, the eyes, eyebrows, node, and mouth are the places with the highest resolution or most points). Once those points are glued to each other, the points on the top layer are completely controlled by the points generated by your face. That’s why when you open your mouth or move your eyebrows the top face also follows them.

The resulting mesh looks like this when mapped onto your face:


In-Depth Look

If you’re really interested in the nitty gritty code of the app, I definitely recommend cloning the repo or downloading the first beta release code. Everything regarding the processing of the face image and the camera face is found in the ofApp.cpp file in the Application group. Remember, you need to have downloaded openFrameworks along with the necessary addons (ofxOpenCV, ofxCV, and ofxFaceTracker) and then throw the code three levels below the root of the openFrameworks directory (it’s easiest to put the app folder in openFrameworks/apps/myApps/. That’ll ensure all of the OF libraries are found and that the project runs correctly. Also, another thing to note is that this project will only run on physical devices because of the use of the front facing camera. You’ll get unknown architecture errors if you try to use the simulator.

What began as a group of about 25 people in a classroom at NYIT has grown to over 1200 members with events being held at various startup spaces around New York—and even once at Viacom. It’s been awesome to watch this community grow and to garner feedback from people across the entrepreneurial spectrum, from the early-stage founders still figuring out their ideas to the seasoned veterans and VCs. Our goal from the beginning was to keep things casual while still offering valuable advice and sparking meaningful discussions. We believe we’ve succeeded, and we thankfully applaud our members—whether they’ve come just once or every month—and all of our impressive presenters. Here’s to the continued success of another dedicated NYC community!

If you’d like to present your startup at a future event or host us at your venue, please email me.

We hope you’ll join us at our upcoming events!

May 13 — Ask Alexis / Neo

June 10 — Scaling a Startup with Gregor Hoffman of 1DocWay

Here are a few highlights from past events: John Pavley [currently SVP of Engineering at Viacom] We met at a gorgeous Viacom event space, where John Pavley shared his experience from working with Apple, DoubleClick, Yahoo!, LimeWire, Spotify, Huffington Post, and Viacom. Pavley is also an Adjunct Instructor at NYU. Check out his brilliant talk on Taboo-Driven Innovation.

OpenBCI An artist and self-taught Electrical Engineer taught a class at Parsons, where one of his students built a brain-monitoring baseball hat. They broke off to fund OpenBCI [BCI = Brain Computer Interface] via Kickstarter and raised $215,000 in their quest for $100k. This was an awesome talk with an even cooler demo!

Schoology This onsite event—i.e. one where Schoology both hosted and presented—was incredibly insightful, as Schoology has done an incredible job scaling a project that began in a college course to support over 7 million users in all 50 states and more than 150 countries.

Casting to the People We gathered again in the gorgeous loft on Bowery that Casting to the People uses as their office—this time to hear about their startup: a mobile auditioning app that lets applicants vote up top submissions, allowing the most talented actors to be discovered—regardless of their connections.

Prior to joining The Hackerati, Jim was the President of SimplyEngage, a SaaS company focused on providing multi-unit brands with the tools and and data insight to engage customers locally. Prior to SimplyEngage, Jim was the Chief Operating Officer of PerformLine (an affiliate marketing and services firm), Automated Resources Group (a marketing services firm focusing on subscriber membership and audience management) and Donnelley Marketing (an infoGroup company) where he was responsible for operating income objectives as well as all aspects of solution delivery services, account management services, data processing services, vendor management, and production services. Prior to Donnelley Marketing, Jim was the Founder and CEO of ResponseLogic, a personalization software and services firm enabling marketers to target web and email content based on a proprietary recommendation engine.

Bringing on Jim as COO is a big step forward in scaling The Hackerati. We’ve been around for almost three years, and most of our growth has been on the engineering side. Jim’s expertise in direct marketing will be pivotal in further expanding our clientele, which will in turn allow us to continue hiring brilliant, creative engineers.

I’ve had the pleasure of working closely with Jim since he began and he’s been tremendously helpful in providing a structured approach to marketing. He’s also helping push Labs forward on the organizational side, including an overview on what it takes to raise capital at our March Pitch Day.

Academy Seminars 

The Hackerati recently launched its first seminar—An Agile Approach to Big Data—through its blossoming Academy. Hackerati Academy has been an idea since our inception and we’re incredibly excited to be expanding the offering. We’ve run several pilot courses in areas like Android, Javascript, and Scala at Cooper Union, NYU Poly, and NYIT. We’ve also used Hackerati Academy as a means of growing our internal capabilities by having periodic Academy sessions during team lunch on Fridays. And now, with the introduction of this seminar, we’re excited to broaden the Academy’s scope and exposure.

An Agile Approach to Big Data debuted on Monday, March 2nd at Turn To Tech. Our goal was to introduce software engineers comfortable in Python and databases to Big Data practices and Agile software development. With support from Amazon Web Services—who offered a $50 AWS credit to all attendees—and BigML, The Hackerati had a blast providing this high-level perspective on practical techniques to talented software engineers.

Hackerati cofounder Rob Marano kicked off the seminar, while Python/Big Data engineers Ethan Lusterman and Derek Toub walked the group through the details. After a lecture overview, these three Hackerati engineers—as well as one from Amazon—gave hands-on advice during a breakout session. We know that the best way to learn is to dive headfirst into problems, so this is the way we teach.

More To Come! 

We’re hosting this event again on Tuesday, March 31st for Cooper Union’s electrical engineering students, and plan to bring it elsewhere in the coming months. We’re also in the process of putting together seminars on some other topics. Keep an ear out for future events if you want to pick up some new skills with Hackerati Academy!

If you’d like to keep abreast of upcoming Academy (and other) events, join our newsletter by filling out the form on the right side of our footer. Inquire here if you’re interested in ramping up your employees’ skillsets with a Hackerati Academy training course.

Back at the UN

The Hackerati was invited back to the United Nations for another presentation to promote Humanicy: The Human Side of Diplomacy. Check out my post on Humanicy’s April 4th media launch event for details on the platform.

December 4th and 5th marked the first full-fledged Humanicy event: a two-day celebration of art and its ability to help us recognize that we are all human beings connected through love and expression. Ambassador Toriello has promoted this idea at the UN because of inherent diplomatic and political buffers between this core human instinct to express and love. As I wrote in my other post, “[Toriello] sees art as the foundation of the human experience, and believes that diplomats and politicians who see each other as fellow human beings before discussing politics are more likely to establish meaningful connections.”

Once again, a question may be quite appropriately raised: how does this involve The Hackerati?

Art is Science | Science is Art

The two-day Humanicy event featured discussions, presentations, and performances highlighting the importance of art, including musicians—like Peter Yarrow from Peter, Paul, and Mary and a Juilliard/Columbia string ensemble—a cartoonist, photographers, painters, a documentarian…the list goes on. Yet there was no focus on the technology driving each of these art forms—at least not until our presentation.

As Creative Technologists—professional engineers who weave their artistic pursuits into their work—The Hackerati teamed up with a few members of Local Projects to illuminate the age-old confluence of science and art. In fact, the word technology comes from two Greek roots: techne, meaning art or craft, and logia, meaning science or study. Everything is technology; everything is a blend of science and art.

I did my best to convey this sentiment in my introduction through my primary method of expression: spoken word poetry.

Grace followed with insight into technology’s influence on heightened empathy. Using open-source facial recognition software, she suggested we would better understand others if we could see oursleves as them.

She took on the face of the Mona Lisa,

the Dalai Lama, 

and even Ambassador Toriello!

Andreas Borg of Local Projects rounded off by using the same software to highlight this empathic response on a global political level, translating facial expressions into sound and asking us to perceive ostensibly cacophonous interactions as orchestrated music. 

You can view our presentation here, starting at 1:45:40.

The Hackerati Band’s Official Debut

To further emphasize the inseparability of science and art, The Hackerati Band—which first came together at our Holiday Party last year—played its first official set the night before, following the Juilliard/Columbia string ensemble. The band consisted of Derby [Vocals and Percussion], Grace Avery [Piano], and Omar Niazi [Trumpet] of The Hackerati, Andreas Borg [Guitar] of Local Projects, and supporting musicians Karen Larson [Violin], Jose Moura [Bass Guitar], and Sebastian Guerrero [Cajón].

While this portion of the Thursday event isn’t on UN Web TV, we’ve posted two songs on our Youtube Channel. Mi Vida was written by Derby and Andreas, and the unofficially titled Grace’s Song was written by Derby and Grace.

Thank You!

TL;DR

The Browser and JavaScripts = SUCH WOW, SO AMAZE!!11!!

In response to: 

Really, Alex? REALLY?

I completely disagree with you, Alex. I believe that browsers are the future of application development, especially when it comes to mobile. When mobile devices first came out, the general strategy was to be the first app in the marketplace to be competitive. But the strategy has shifted to differentiating an app with better performance and UI, which is mainly achieved through native libraries.

In the near future, users will increasingly own heterogeneous devices across a range of manufacturers, with wearables and other connected devices entering the mix. Users have come to expect a seamless app experience across all of these devices and continuous updates throughout the lifetime of the app. The only way to deliver this at the speed of today’s trends and stay competitive by covering all of these devices is to use a web-based strategy by leveraging existing browser technologies into applications.

Browsers were originally designed for delivering static HTML pages, but I believe we are seeing a round of new thinking towards browsers running modern applications. While most browsers have traditionally been CPU-intensive, frameworks and technologies are tackling app performance by taking advantage of GPUs for UI (see WebGL and Famo.us) and rethinking how DOM manipulations occur (see react). Below are some links to applications being built on browsers:

https://www.mobilespan.com/content/chrome-is-the-new-c-runtime

https://github.com/rogerwang/node-webkit

https://atom.io/ (Built on top of webkit; takes advantage of JavaScript/CSS/HTML for extensibility/plugins)

As for JavaScript, I really can’t imagine a more exciting language right now. Whole operating systems are being developed to run exclusively HTML5/CSS/Javascript apps (Firefox Mobile OS, ChromeOS). It’s easy enough to get started for beginners, and now that you can build more than just web frontends, it has really taken open source to the next level with the average layman releasing a new library every week.

The depth of the language is now evident with its prominence in the reactive/asynchronous programming and functional programming movements. The fact that you can run JavaScript code written by a stranger without having to install it or care too much about security is unique, along with its ubiquity in terms of developer talent and its ability to run on any platform with low overhead.

Even if you don’t like the particulars of the language, you can take still take advantage by leveraging projects like Emscripten to compile your favorite language into JS bytecode. Below are some impressive examples of what the future of JavaScript might hold:

http://vps2.etotheipiplusone.com:30176/redmine/projects/emscripten-qt/wiki/Demos: Qt GUI framework running in your browser

http://kripken.github.io/sql.js/GUI/: SQLite Database in your browser

http://www.quakejs.com/: Quake 3 running in your browser

http://github.com/kripken/emscripten/wiki: More examples of projects converted to JS

http://bellard.org/jslinux/: Linux OS in your browser (not Emscripten—straight JS implementation)