thetatao potatos

This weekend was Santa Con 2017. Woo! It was also DLSG XV (deep learning study group 15) put together by Jon Krohn where we learned about reinforcement learning. I'm summarizing it here.

High Level 

Reinforcement learning is an approach to ML type problems that has become popular. Reinforcement learning is often applied to Atari games like pac man or ping-pong. The framework it uses is different than other kinds of deep learning methods. Instead of simply having an input and an output through some neural network architecture you consider the problem from the point of view of the player or Agent. The agent can interact with its world in varying degrees of discrete or continuous decisions. These decisions or Actions lead to new states and cause the agent to succeed or fail at its objective. This success is a measurable reward which is also used to then update the policies that the agent uses to make future decisions. Actions can be as simple as moving up down left or right and states are like the position of the agent and reward is the score of the game or whether or not the agent dies / loses. 

Deeper

Reinforcement learning can happen in different ways and there are several algorithms that do this. A Q-function is used to describe what the agent should do. It is the function that keeps all the policies the agent will use in the game or environment. A neural network is used to approximate the Q-function. A perfect Q-function indicates that the agent knows how to act optimally. π(s) = maxQ(s,a). Policy (pi) as a function of state equals the max Q function as a function of state and action. Q functions answer how good is a state action pair while a value function answers how good is a state

A = Q - V

A couple months ago I went through the Capital One Interview process for Data Scientist. Part of the interview was a take home with two parts. The first part was to perform a regression model on anonymous data, with 4 categorical columns and over 190 continuous ones. The first thing I did was look at the data. 

Exploration

It’s nice to see if the columns are normally distributed before continuing. A lot of techniques assume a normal distribution. It’s also nice to see if there are any particular columns that correlate closely with the column we wish to predict. 

We can see that one particular column (labeled 175) stands out. When we plot target vs this column, we see that the points fall on an elongated ellipse. Compare this to another uncorrelated column as above on the right, and it looks like a set of random points in a circle. 

In pandas, we can run a Pearsons correlation and look at other columns that seem relevant as well. As you can see. There are multiple columns that are related to one another. In pandas, run .corr on your dataframe and then unstack and sort by correlation value to build this new table. 

Imputing and One Hot Encoding

We have a slight problem with going straight to building our regression model. The data has NaN’s and categorical features. we must deal with the missing values in a smart way and also transform the categorical features so that they can be incorporated into the model. 

There are several ways to solve the problem of missing data. A quick once over shows that NaN's make up and they're distributed randomly in the dataset. There is a library called fancyimputes that makes it easy for us to fill in the missing values using K nearest neighbors.

We use pandas get_dummy to one hot encode the categorical features. I built out a library to take care of this called morph_data (which is what it is called in the github repo). 

Building out Models

Its time to create a model. The data has already been broken into a training and test set but unfortunately our test set doesn’t contain the value we wish to predict. In order to explore possible models we split the data (our training set) into a test and train set. You can do a 60% train 40% test split or some variation. 

Linear Regression

Linear Regression with elastic net regularization gives us an R Squared 0.6153. However, the higher the R Squared value the better our model fits the data. Even though I used grid search for the right parameters using the sklearn library, I wasn’t able to get this model to perform better. Therefore, lets try something else.

xgboost

Lots of people have been using extreme gradient boosting to build models so I also looked at this. This model is like averaging lots of tiny random forest models. Refer to github to see how I used the library for regression. We use the mean squared error as the log-loss function. The R Squared is now 0.8134. Great! But not good enough. 

MARS 

From reading, it’s much easier to implement a multivariate model in R than in python. I found a library called pyearth with multivariate adaptive regression splines capabilities, which I use here. Part of getting these models to work is giving them the appropriate parameters. The important parameter for this model was setting max_degree=4. The R Squared is 0.9404 and I’m happy. 

Summary

You can see how well the models now predict the correct result. The more linear the points below the better the model. 

Turns out conspiracy theories about Jesus are about the only kind of conspiracy theory that has equal amounts of positive and negative words. All other conspiracy theories are shrouded in negative words. I should probably look at a larger data set before drawing any conclusions. 

Today I attended the Monthly Music Hackathon at Spotify and learned some circuit building! The chip in the center is a 555 time chip. The potentiometer controls the resistance in the circuit which changes the current through the 555 timer chip. The higher the current, the higher the pitch of the speaker in the bottom left of the board. 

Picture of the circuit on prototype paper: 

Here is a schematic of the circuit:

Here we look at the Yelp data set, which provides information about user ratings for businesses in several large cities across Canada and the US. Since the dataset is quite large, I’ve only used 10% for this analysis. The first thing we notice is that most reviewers give 5 star ratings and very view give 2 star ratings. But why?

When we look at the sentiment associated with each score we immediately see that a 2 is the point at which reviewers use positive and negative words equally (NOT 3 as one might expect). To calculate these values, I take the normalized sum of positive and negative words in each review. The positive and negative words were built from preset curated lists. 

If you look at the percentage of reviews by star rating you start to see that peoples ratings diverge over time -- but why?  (results are cut off: deep blue is 1, green is 2, red is 3, aquamarine is 4, and purple is 5.) We see the percentage of 5 star and 1 star ratings increase while the percentage of 4 and 3 star ratings decrease. 

Using a logistic regression classifier we are able to predict the users rating from their review text using the positive and negative sentiment scores. We can distinguish 1 star ratings and 5 star ratings with 88% accuracy. 

This analysis however, does not take time into account. It’s clear that there is a trend in the result. But is this increase in percentage of 1 and 5 star ratings pinned to the actual feelings of reviewers? Is sentiment more polarized over time or are users simply voting more 5′s and 1′s because they want their opinion to weigh more? We can explore this by checking if review sentiment stays constant or changes over each year. 

By resampling the sentiment by month for both the 1 and 5 star ratings and their respective positive and negative word counts, we see a few things. 1 star ratings have higher negative sentiment and 5 star ratings have higher positive sentiment. What’s interesting though is that 5 star ratings actually become more positive each year. All other attributes stay constant. This increase in positive words is also true for 4 star ratings and slightly so for 3 as well. In general, people have been writing reviews with more positiveness over the years.

Link to Presentation 

I wanted to build a classification model to test if part of a genome was actually a gene or not. I started by splitting the genome into segments of 50 but quickly realized this wasn’t a good start. I found genes by identifying where they could possibly start and where they could possibly end. Almost all genes start with the nucleotides A-T-G and almost all genes end with T-A-A. Of course many of these overlap. I made a compilation of all the potential gene starts and then took 200 nucleotides downstream of that sequence and stored all of those in a pandas dataframe. I used an annotated genome to label those segments are ‘gene’ or ‘not gene’. I then built a binary classification model around that. Logistic regression worked pretty well with a 90% accuracy score. I made sure that my dataset was balanced with the training set - so the worst accuracy possibly would have been 50%. 

Here’s what the genome of Streptococcus pyogenes looks like when you map out the forward genes as blue and the reverse genes as green and the empty genome regions as red. 

I scraped from some sites like IMDB and got huge but incomplete data sets to run a regression model to predict movie revenue. 

We find that production budget along with the top two actors are pretty good indicators of opening night movie revenue. R^2 = 0.71 before I learned how to optimize things. I wanted to use google trends to help with the prediction and make a measurement of how often people searched for that movie title but didn’t finish with that. There are some issues since google trends provides relative values. In a research lab I’d use a standard like the word ‘the’ or ‘is’ which shows up everywhere. 

I also looked at the movie topics over time. It was like 11pm the night before the presentation was due and i webscraped all the > 1000 rated movies on IMDB along with their top three genre categories. We see a sharp increase in the number of rated movies -- likely earlier movies are less rated but also likely - we’re producing SO MANY more in the last 15 years than we ever have before. 

Most movies are labeled as drama or comedy. Although, comedy only becomes a dominate category in the early 1960′s. Before then, It looks like romance after drama, dominated. 

Dip in Horror Genre in 1990s (WHY?) -- we also see that in 1944 - 1945 there is an increase in both mystery and crime movies. Is this related to WWII?

This weekend I went up to Boston for the Open Data Science Conference (ODSC). Friday morning Bang Wong, Creative Director of the Broad Institute of MIT and Harvard, and Mark Schindler, co-founder and Managing Director of GroupVisual.io, gave an introductory crash course into data visualization. They used the Hubway bike system data set as a tool to teach the audience about visualization decisions. 

The dataset seemed cool so I decided to investigate gender differences in Hubway bike users. Firstly, here’s what the data looks like. In total, males spend 3x as much time using the bikes as females do. 

The periodicity in this dataset is because people use the bikes less on weekends. Don’t believe me? Here. 

Most rides happen during the week! But I found something kind of cool, but also obvious about this. Below you’ll see that although bikes are used for more time and more often during the week, ride duration is higher on the weekends and not only that, women on average spend more time than men on bikes. What this means is that females bike slower than males and weekend rides are generally more leisurely. 

We can aggregate week times together to see some interesting day and week patterns in Boston. The first number represents the day of the week where 1 is Monday.  Second number represents the day hour ( 0 - 24 )

I decided to compete in the google code jam. Below is the answer to the first problem, which you can find here. The problem was to take something like input 'zwetroo' and return '02' because the letters for zero and two are in that string. (.eg input 'ifvoferoeuzr' would have output '045'.) I first looked for numbers with unique letters in them. For example 'zero' is the only letter representation of 0 that has the letter 'z' in it and 'four' is the only letter representation of 4 that has the letter 'u' in it.

I recently stumbled across a tumblr blog kept by Bill Wellington. If you haven’t heard of this data scientist dude - check out his blog and his ted talk. Taking his lead, I’ve decided to inject some ‘data analysis’ like posts here. SO. Github has a developer api for pulling data related to the number of commits made in repositories. If you’re familiar with Github, you’ve probably stumbled across the ‘graphs’ tabs and looked at commit / contributor data. You can pull that data as a json. 

I asked the question: When are most commits made during the week? To do this, I looked at a few popular repositories, split each day into working hours (9am to 5pm) and after hours (before 9am and after 5pm), and used the punchcard hourly data from the github developer API in some stacked bar plots.

Data here:  https://api.github.com/repos/AlexNisnevich/untrusted/stats/punch_card

Data here:  https://api.github.com/repos/torvalds/linux/stats/punch_card

wget

I'm always looking for ways to improve my efficiency. Because I often work in coffee shops with questionable internet access, my downloads run slow and time out. They get cancelled every time I lose connectivity. So I came across a potential solution called wget.

wget installation -- to help complete paused or interrupted downloads (save time) etc.

Download here: http://coolestguidesontheplanet.com/install-and-configure-wget-on-os-x/

How to use it : http://www.gnu.org/software/wget/manual/wget.html

eg. right click on the link to the download and then in terminal type wget linkurl If you want to deviate from the default, look at the gnu.org manual. The norm is 20x attempts, use -t inf for unlimited tries and & after the link name so it runs in the bacground. wget saves the file in the directory you navigated to in the terminal.

URL is an acronym for Uniform Resource Locator

dragdis

$.get() requests are like $.ajax() requests. I think you can do the same thing with both, only that get is ONLY for retrieving data, whereas $.ajax() can also update the database. This is a short post just explaining the two main ways data pulled from a $.get() request can be used.

The information is used immediately in a callback. Most examples out there are like this. You invoke a callback method that uses the data immediately. For example, an alert or some sort of page render.

If however, you want to save the data so it can be used whenever you want, this is the way to do it.

```Javascript $.Carousel.prototype.getFolders = function(){ var that = this; var this.$storage = []; $.get("/img/art/", function(data){ $(data).find("a").each(function(){ that.$storage.push($(this).attr("href")); }); }); }; ```

if we want to use the data outside of the $.get request, we must store the data in a variable. Regular variables are only available inside the function they are declared within. SO that won't work.

However, instance variables are available throughout the entire class. Instead of saving data to var $storage, we instead save it to this.$storage. We store data inside an anonymous callback during the get request.

To keep correct scope, store 'this' in a variable called 'that' within the Carousel class method getFolders. Because all functions called within this method 'close over' (closure) variables declared within this method, we still have access to them inside the callback.

Jquery (JS) library makes it easy to build an interactive webpage. Ok, if you've heard of it, you already know that. But how? Here are some fundamentals that will make learning Jquery easy. I hope. :).

Firstly. You're here because you want to make a dynamic and interactive webpage. Right? That is, css and html changes, as the user interacts with the page. Well, you don't need jquery to do that. javascript already does EVERYTHING jquery does. However, jquery makes it simpler. If you want to dynamically update a database from user inputs, you need AJAX. Which I will write about later.

You need to know what a DOM (document object model) is.

Take a look at: https://developer.mozilla.org/en-US/docs/Web/API/Document_Object_Model/Introduction.

"The DOM provides a representation of the document as a structured group of nodes and objects that have properties and methods"

Wikipedia has a few tables listing the type of events that you might want to retrieve. https://en.wikipedia.org/wiki/DOM_events.

bubbling vs capturing. Bubbling! = an event listener placed on a parent html element will be captured on any child element within the parent. http://javascript.info/tutorial/bubbling-and-capturing

JS allows you to listen to events that occur on HTML elements within the HTML document. Events can be handled in two ways. Either both bubble and capture (parent down to child). jquery ONLY allows bubbling (child up to parent)

jquery .on() vs JS .addEventListener()

jquery allows the developer to turn html elements into objects with attributes and properties.

For example, the entire document (html page) can be turned into a jquery object with $(document). A ul with class name "schools" is a jquery object if written as $(".schools").

The browser records everything, it seems. User clicks, mouse location, key strokes, etc... Jquery gives us a way to listen to whether those events have happened. When they do happen, we can gather information like event location, timestamp or target, etc... and use it to make css and html changes to the browser.

A listener looks like: $(html element name or document).on(event type (click / keydown / keypress , event location (an element name), event handler (a function with access to the event)).

The listener can look for changes to particular elements in teh DOM (document object model) Take a look at: https://en.wikipedia.org/wiki/DOM_events

http://stackoverflow.com/questions/4616694/what-is-event-bubbling-and-capturing

https://developer.mozilla.org/en-US/docs/Web/API/EventTarget/addEventListener

I just completed 6 weeks of App Academy. This is 6 out of the 7 weeks of curriculum and half of the App Academy program. It is also my final week in the program. However, there is a huge silver lining. I went through almost the entire curriculum and can now have the free time I need to finish projects I started while studying in the program. They will return my deposit and I'm no longer obligated to find a job in NYC or give them 18% of my salary when the program is complete. Let's see if I can push myself to solidify what I've learned and put it to good use in the job market.

I'm not sure I really wanted to become a web developer -- as that is what the program is designed to teach. It's a fullstack curriculum that provides students the tools to do anything within the entire spectrum of web development: from back end like database work, to front end, like implementing ajax forms and sculpting out the css to make websites look and behave pretty. I was initially drawn to the program because of the effort I had to put into studying for the interviews and the allure of how exclusive acceptance is, along with the average salary students coming out of the program were landing.

The students in the program were / are stellar. I definitely learn ... slower than the other students. It takes time to retain vocabulary and understand the language framework. Having other students in the program explain things to me, made learning easier. I dont know if I could have done it myself! We were paired each day so I also had a chance to see the standards other students in the program held themselves to while studying and learning new material. Speed was / is important for web developers.

I've always been drawn to seeing how people learn. How we form the framework to understand concepts etc. So this experience was amazing for me. No one in the program had a lazy brain. Everyone was constantly connecting concepts and absorbing the new material. Programming is very much a language skill. You have to learn the grammar and the vocabulary just like with spoken language.

I feel free right now, because it means I can build my own schedule for the next couple months while I prepare for applying for jobs. My plan is to study. I'm writing out in detail what I want to achieve each day, looking ahead by no more than a single week. Linda, one of my friends, is going to help me make sure I stick to the plan I make. Today is reviewing DFS and BFS.