Come as You Are...

A sampling of projects that Hackerati folk work on in their free time!

Agile engineering is just as important to streamlined software delivery as agile project methodologies. 

Here’s an end-to-end, fully functioning (albeit trivial), automatically testable and deployable Node/Express project that represents only one point of view of what’s good and what works for a basic Web or micro-services application that supports an agile engineering workflow. This code is MIT licensed, so please use it, critique it, fork it, and help to improve it!

Here’s the repo:

https://github.com/thehackerati/node-app-template

What’s Inside

Docker is the heart of this environment.

For local development on a Mac, we rely on Vagrant to provide a Linux virtual machine within which we can run the Docker images. If you use Linux for development, you can run docker-compose directly.

The development environment allows you to edit source in your favorite text editor or IDE in your development host, while the application automatically runs tests and reloads inside the Docker container.

Travis-CI automatically runs tests and deploys builds to staging and production environments when you push and merge code on Github. You can see the build running here:

https://travis-ci.org/thehackerati/node-app-template

Staging and production environments run Docker images on AWS Elastic Beanstalk. You can see the code running here:

http://nodeapptemplateprod.elasticbeanstalk.com/

http://nodeapptemplatestaging.elasticbeanstalk.com/

There are two permanent branches in this repo:

master: automatically deploys to the production environment on push or merge to master

staging: automatically deploys to the staging environment on push or merge to staging

Setting Up the Environment

To contribute, you can start by forking this repo and cloning your fork to your local development environment.

To use this template as the basis for an application, clone and rename it, and push to a new repo on Github.

Once you have a new repo on Github and a local clone on your development machine, you can setup the rest of the environment:

Create automated build on Dockerhub

Connect Tavis-CI to Github repo

Run scripts to setup staging and production environments on AWS Elastic Beanstalk

Run vagrant up

See the README for detailed instructions.

Using Principal Component Analysis on Images

Any set of images can be broken down with Principal Component Analysis. This has been done pretty successfully with faces. Here we’ll take a look at style.

Our dataset is 807 pictures of dresses from Amazon. They have a standard image size, but unfortunately do not have a standard model pose (though they tend to be centered in the image similarly). Ideally, our principal components would only be about actual dress style, but here many of them will be concerned with model pose. Despite this, we can still do a lot with this data set.

This eigendress is the first principal component, which accounts for the most variation among all the dresses. Broadly, it’s looking at light-colored dresses vs dark-colored dresses.

The second component seems to look at short dresses vs long dresses.

Reddish colors vs blueish colors.

Short hair and sleveless vs long hair and long sleeves.

Posing with legs close together vs posing with legs farther apart.

And so on.

Using components to recreate images

With a bunch of components like these, we can reduce an image from, eg 60,000 points of data (pixel values) to just a handful of numbers.

Let’s recreate this dress from its components.

The following pictures are created from one, two, four, nine, ten, fifteen, thirty, forty, and seventy components (respectively).

As you can see, the more components we have, the more accurate and detailed the dress recreation will be.

The data for the middle dress above now looks like this: [-17541.81, -12749.33, -3766.29, 2005.28, 4193.08, 6832.55, -6704.90, -2135.51, 1112.27, 7627.80].

So, if you have a million pictures you need to store, you can save a whole lot of space by saving just the component values instead of the values of every pixel of every dress.

Using components for prediction

I’ve also manually categorized the pictures as dresses I like (287 pictures) and ones I dislike (520 pictures).

Now we can use logistic regression on component data to predict whether or not I’ll like a dress.

Sorting all the dresses by score, it can show the prettiest and ugliest dresses of the whole set.

The prettiest dresses:

The ugliest dresses:

Seems pretty spot on! I could now set up something to watch as new dresses are posted on Amazon, and to alert me to dresses it thinks I will really like.

Creating new dresses

For creating pictures, there’s no reason we need to confine ourselves to already known dress component values. We can also choose random values for each component, and see what happens!

Completely new dresses! With more data and better data, this could actually be a viable dress design tool!