Ryan Musser

Updated on 4/12/17 to include traffic, reach and usage statistics.

I had an absolute blast building this project! I wrote this blog to explore some brief behind the scenes aspects of the project. Enjoy :)

BasketballRoto.com is a tool designed to help daily fantasy basketball players optimize their lineups based on advanced analytics, up to the minute news and statistical projections. It is a free alternative to websites like:

FantasyLabs.com ($69.99/mo)

BasketballMonster.com ($100/season)

NumberFire.com ($49.99/mo)

RotoGrinders.com ($25/mo)

 I have not monetized the website.

Tech stack

I built BasketballRoto.com using Meteor (a Node.js framework), React, Redux, MongoDB, webpack, babel, D3.js and my genetic algorithm. It’s hosted on a 2GB Digital Ocean droplet. I deployed the app using meteor-up.

The lineup optimizer: algorithmic muscle

I wrote and published a compounding genetic algorithm to solve the lineup optimization pairing problem. 

You can see the NPM package here and an in depth write up on the theory and benchmarks I observed when writing the package here.

User acquisition

I built a Twitter bot (@BasketballRoto) that compiles NBA updates, adds relevant hashtags and @ users, then tweets them out every 31 minutes. The 31 minute interval is in compliance with Twitter API usage guidelines. In the first month, the bot garnered 300+ followers and has driven the vast majority of traffic to the site. Aside from showing a few friends, this is the only form of user acquisition I have used to promote the site. Here’s a screenshot of the bot’s performance from 12/7/16 to 1/3/17:

You: Cool screen shot of your Twitter dashboard, bro. What does that mean in terms of site traffic?

Me: Well according to Google analytics over that same period:

Update on 4/12/17 - See the following usage statistics for the entire first NBA season from 12/7/16 to 4/12/17.

Daily users

Site usage

Twitter impressions

The content: projections, tweets and robo-blog

Projections can be updated every 16 minutes, depending on whether or not our data provider has new projections to offer. The website updates itself live; no page refreshes needed, Projections are fetched via GET request; I store a hash of the current projections in the DB. Every 16 minutes I compare that hash to a hash of what’s returned from the GET request to our data provider. If it’s different, I update the database with the new projections. If not, no action is taken.

Tweet content is harvested from other popular NBA based Twitter accounts. Tweets are parsed, applicable NBA relevant hash tags and @ are added, then tweeted out by my bot. This process runs every 31 minutes so as to be compliant to the Twitter API guidelines.

The Sleeper Report is an auto-generated blog that is updated each time the NBA statistical projections are updated.

It costs me roughly $300 annually to keep this site running and it’s completely automated.

First off - thanks for reading, I do appreciate it. I wrote this article with the intention of it being read by technical, non technical and fantasy basketball agnostic readers. Soooo there’s no code in this article. If you want to see that, I open sourced my solution on NPM. Enjoy :)

In extremely short summation; this is about how I created a compounding genetic algorithm to solve a pairing problem with 17,437,197,465,249,097,536,000 potential pairs with an average of 98.54% accuracy in around one second.

Ok - so let's dive deeper into the problem I wanted to solve.

The Problem

FanDuel is a popular daily fantasy sports site; I’ll be focusing on the NBA game that FanDuel offers. In the game, the user is given an imaginary budget of $60,000, and asked to choose a lineup from a pool of NBA players. There are certain constraints on the game:

Positional constraints: The user can select 2 point guards, 2 shooting guards, 2 small forwards, 2 power forwards and 1 center. Each player can only play one position and you can’t select a player twice.

Budget: Each player is given a $ cost by FanDuel, and in selecting their lineup, the user cannot spend more than $60,000 in total on their players.

Let’s take a look at the interface to help visualize what the user sees:

In any given night there can be upwards of 300 players in the player pool. Positional and budget constraints aside, when choosing a combination of 9 out of 300 players, there are roughly 17,437,197,465,249,097,536,000 combinations possible, which is 230 times more pairings than grains of sand on the earth. So - if I wanted to write a program that searched for the valid highest pairing - you can imagine that searching for the pairing using an inefficient algorithm would result in an absurdly slow processing time (Much longer than your lifetime).

Enter genetic algorithms.

In short, a genetic algorithm finds a solution based on a natural selection process, mimicking biological evolution. The algorithm repeatedly breeds and mutates a population of possible solutions based on each solution’s fitness. Fitness is pre-defined by the engineer and evaluated by the program she writes. Genetic algorithms can be constrained (we know the ideal outcome) or unconstrained (we don’t know the ideal outcome).

In this case, the genetic algorithm I built would be considered unconstrained. I don’t know the lineup with the highest projected statistical output before processing. This means I must write the program to stop itself after a certain amount of processing time, therefore the program won’t necessarily hit the highest possible pairing by the time it’s terminated.

Now - I like to be as efficient as possible in my work. So, after documenting the problem I wanted to solve and deciding the way I wanted to solve it, I scoured the web in search of a solution that had already been coded. (Also, for the technically inclined, I wanted the solution to run in the browser, which essentially means it must be written in javascript) And as you can probably guess since I’m writing a blog on how I wrote the solution myself, I didn’t find any pre-written solutions.

Genetic Algorithms: My Brief Adaptation

Now, as a heads up for the not so technically inclined, we’re going to dive a little deeper into the algorithm, the theory behind it and some benchmarking from my tests in developing my solution. I enthusiastically invite you to stick around and read on - I think I did a pretty good job of explaining to all audiences ;)

So - The diagram below gives a great high level overview of what the process looks like:

Moving forward, I am going to use this diagram as an outline to describe each portion of my algorithm. 

These types of algorithms are called genetic/evolutionary algorithms because, as you can see above, they directly mimic natural selection. The diagram depicts what I like to call an evolution, and within that evolution you see an area labeled Generation cycle, which I will describe as a generation moving forward.

In the diagram you also see a Start --> Randomly generate initial population section. In my algorithm, this consists of randomly spawning 200 lineup pairings. In order to help the process along, this process involves ensuring that the lineups are positionally sound (Have 2 PGs, 2 SGs, 2 SFs, 2 PFs and 1 C). That is the only pre-validation, as the majority of ‘validation’ is handled in the next step; Evaluating all individuals.

Once the initial generation of 200 random lineups is spawned, as you can guess, we must evaluate the fitness of each lineup, which is shown as Evaluating all individuals in the diagram. In evaluating the fitness of a lineup, we look at the following:

Does the lineup meet budget criteria? If not, the lineup is considered unfit and given a fitness of 0.

Does the lineup meet positional criteria? This is a double check, but if it doesn’t match, the lineup is considered unfit and given a fitness of 0.

Do we have multiples of the same player? If so, the lineup is considered unfit and given a fitness of 0.

If the lineup passes the tests above, it is considered fit, and its fitness is the cumulative projected fantasy stats of each player in the lineup.

Now, this is where the magic happens! Are you excited?!

Once each lineup is evaluated and given a fitness, we select the two fittest lineups to breed and mutate into the next generation.

Breeding, in this case, simply means that we’re creating 18 new lineups. Each lineup is a replica of it’s parent lineup, with one player from the other fittest lineup swapped in. For example:

If parent 1 looks like this:

A,B,C,D,E,F,G,H,I

And parent 2 looks like this:

J,K,L,M,N,O,P,Q,R

Then some children of breeding these two pairings would look like this:

J,B,C,D,E,F,G,H,I

A,K,C,D,E,F,G,H,I

A,K,L,M,N,O,P,Q,R

J,B,L,M,N,O,P,Q,R

etc.

Now, mutating, in this case, means something entirely different than breeding. The number of mutations isn’t pre-determined by the algorithm, the user of the algorithm can decide how many mutations they want (as it slows the process but helps reach higher fitness levels). Each mutation is a replica of it’s parent lineup, with one player from the lineup randomly selected from the player pool. For example:

If parent 1 looks like this:

A,B,C,D,E,F,G,H,I

Then two mutations of this lineup may look like this:

Z,B,C,D,E,F,G,H,I

A,B,C,D,E,F,W,H,I

Now after breeding and mutating our population, we have created a new fitter generation to repeat the process with.

Once the predetermined amount of generations is completed, the fittest lineup is stored, which completes a single evolution. From each evolution, the fittest lineup is stored, and once the user-defined amount of evolutions is finished, the fittest lineup is returned. This is considered a compounding genetic algorithm as it starts each evolution with a clean slate. So, if the user decides on 10 evolutions, then once each evolution is complete, the program will select the fittest lineup from those 10 evolutions and return that lineup.

Now, let’s talk about my experiences with performance, incest and tweaking the algorithm! (Everyone’s favorite subjects!)

Parental Incest

Incest is suboptimal - and in quick summation, I’ll show you why :)

In this case, I consider incest to be a reintroduction of the parent lineups back into the next generation breeding pool. I ran 10,000 tests to observe the influence of incest on my algorithm, and here are the results:

Without compounding and with 40 generations, when the parents were reintroduced into the next generation’s breeding pool for 5,000 tests, the algorithm produced and average fitness score of 300.4118

Without compounding and with 40 generations, when the parents were removed from the next generation’s breeding pool, when the test was ran another 5,000 times, the average fitness score was 301.484

Not a crazy performance boost, but a performance boost none the less!

Sibling Incest and the False Plateau

After around 10 generations, I noticed that the two fittest lineups would tend to be the same. We would be breeding the same lineups, which created the same lineups. This essentially relegated all the magic of evolution to mutation. This is not optimal and is an effect I also classified as a symptom of algorithmic incest.

This created what I call a false plateau, and hindered the algorithm from continuing it’s evolutionary adventure. Think of it like this, if you put a blind man at the bottom of a hill and asked him to keep walking up the hill until he hit the top, what if he hit a plateau? Theoretically, he would believe he was at the top, and not want to venture down to far away from that peak.

My algorithm was suffering from this type of issue by breeding the same two lineups. So, to fix this, I ensured that the two fittest lineups could not be the same. They must contain a different set of players. The performance results were beautiful!

Without compounding, with 40 generations, not reintroducing the parents back into the breeding pool and ensuring that the two fittest lineups were different each generation, 5,000 tests yielded an average fitness of 306.672

That’s a huge boost! But wait, there’s more...

Compounding Evolution!

Compounding evolution sounds way more complicated than it is. This simply means allowing the algorithm to run once, save it’s top lineup, then run again. Each one of these cycles stores it’s top lineup, hence the name compounding. Then, after X evolutions, the overall top lineup from those X evolutions is returned. 

So now - with 40 generations, not reintroducing the parents back into the breeding pool, ensuring that the two fittest lineups were different each generation, and picking the top lineup from 5 evolutions (40 generations 5 separate times) - after running 5,000 tests the results yielded an astounding average fitness of 313.426 all in around 1 second each time!

Boom baby! For reference, with this dataset, the fittest possible lineup had a fitness of 318.069

Wrapping up this article...

I hope you found a little joy in exploring and possibly contributing to or using my project! I do appreciate your time very much. If you’re looking to see this algorithm in action, go ahead and optimize a couple lineups on my website at BasketballRoto.com and if you’re interested in seeing the code, check my GitHub repo or the NPM package. Thanks!

Prerequisites:

You're familiar with the concepts covered in How do Javascript closures work? A basic introduction...

You have a basic understanding of object oriented programming.

At this point, you should have a low level understanding of Javascript closures. You see how closures can be used to create and freeze new chunks of code with their own local variables, but maybe it's not entirely clear how this is valuable or what this has to do with object oriented programming. Well my friend, this is where the fun part begins!

In this article, we're going to explore how closures and objects can create the data structure and methods to maintain a very basic client side comment thread. Before we get started, I should note that we will not be building an interface for this example, nor will we be discussing any server side aspects, we will only be exploring the underlying javascript data structure and it's associated methods.

In this example, we are going to call our comment thread a wall. In our wall, we will have comments, and those comments will have associated data (upvotes, replies, etc.) Let's create our wall object and some very basic associated methods:

function initiateWall () { var commentThread = []; /* this is an enclosed empty array that will hold the comments on a newly initiated wall object */ return { showComments: function () { /* a method to show comments on the wall */ return commentThread; }, addComment: function (string) { /* a method to add a comment */ if( commentThread.push(string) ){ return true; }else{ return false; } }//end of addComment };//end of return }//end of initiateWall var ourWall = initiateWall(); //we create our new wall object ourWall.addComment('comment 1'); ourWall.addComment('comment 2'); ourWall.addComment('comment 3'); //we've added three comments console.log( ourWall.showComments() ); //the output is an array with our 3 comments //['comment 1','comment 2','comment 3']

Now, the above example is a good start. We can initiate a wall object, add comments to it and access the array of comments in it's entirety. But what if we want to access a specific comment or it's properties? In this case, an array isn't the optimal data structure for the job. We're going to change commentThread to an object, so that we can use key : value pairs to easily access a comment using a hashed unique identifier (the unique hash may come from the server, a hashing function, etc. that's entirely up to you, but for this example, I'm simply going to make the hashes up) Let's take a look at our updated function:

function initiateWall () { var commentThread = {}; /* we are now initializing our comment list as an object itself within the wall object. We do this so that we can reference keys within the object, instead of indexes within an array */ return { // // showComments: function () { return commentThread; }, // // addComment: function (hash, string) { if( !commentThread[hash] ){ /* make sure the comment hash doesn't exist already */ /* let's initialize our new comment object and give it some basic properties */ commentThread[hash] = {}; commentThread[hash].comment = string; commentThread[hash].upvotes = 0; commentThread[hash].replies = []; return true; }else{ return false; } }, // // showSingleComment: function(hash){ /* We've added a new method here that will give us access to the properties of a single comment within the commentThread by passing it the comment's unique hash */ return commentThread[hash]; } // // };//end of return }//end of initiateWall() definition var ourWall = initiateWall(); /* now, we'll pass in some fabricated hashes and coinciding comments */ ourWall.addComment( 'xh7w3' , 'comment 1'); ourWall.addComment( 'yd5b1' , 'comment 2'); ourWall.addComment( 'nd9p6' , 'comment 3'); console.log( ourWall.showComments() ); /* displays list which contains all of the comments in our wall object */ console.log( ourWall.showSingleComment('yd5b1') ); /* displays a single comment, referenced by the unique hash we've assigned it */

Alright sweet! We've got a pretty cool working example of how we can use a Javascript closure to initiate a wall/comment section. As you may be able to see, we can continue to add methods to our object to increase the complexity of our program. For example, I could add the following to initiateWall () to handle replies and upvotes:

// // addReply: function (hash, string) { if( commentThread[hash] ){ /* make sure the comment hash exists */ commentThread[hash].replies.push(string); /* and add the reply to our array of replies */ return true; }else{ console.log('Comment with hash '+hash+' does not exist!'); return false; } }, // // addUpvote: function (hash) { if( commentThread[hash] ){ /* make sure the comment hash exists */ commentThread[hash].upvotes++; /* and increase the upvote count */ return true; }else{ console.log('Comment with hash '+hash+' does not exist!'); return false; } } // //

We've come a long way from How do Javascript closures work? A basic introduction...! If you're not quite grasping these concepts, I highly encourage you to copy the example above and modify it on your local machine or a JS Bin. Try instatiating a wall object using initiateWall(), and mess around with invoking it's methods. As an outro example, I'll add some final code below, and the ouput that it will give you:

var ourWall = initiateWall(); /* We'll pass in our fabricated hashes and coinciding comments */ ourWall.addComment( 'xh7w3' , 'comment 1'); ourWall.addComment( 'yd5b1' , 'comment 2'); ourWall.addComment( 'nd9p6' , 'comment 3'); console.log( ourWall.showComments() ); /* The output is as follows: [object Object] { nd9p6: [object Object] { comment: "comment 3", replies: [], upvotes: 0 }, xh7w3: [object Object] { comment: "comment 1", replies: [], upvotes: 0 }, yd5b1: [object Object] { comment: "comment 2", replies: [], upvotes: 0 } } */ /* let's add a couple upvotes and a couple replies to this comment */ ourWall.addUpvote('yd5b1'); ourWall.addUpvote('yd5b1'); ourWall.addReply('yd5b1','Great comment!'); ourWall.addReply('yd5b1','Very average comment!'); /* now let's see what the 'yd5b1' comment object looks like within ourWall */ console.log( ourWall.showSingleComment('yd5b1') ); /* The output is as follows: [object Object] { comment: "comment 2", replies: ["Great comment!", "Very average comment!"], upvotes: 2 } */

Prerequisites:

A basic understanding of programming structure.

A basic understanding of Javascript.

Bonus: A basic understanding of object oriented programming.

Javascript closures are a beautiful thing. The beauty of closures is that they are only as complicated as you make them - and when wielded properly - they can give your program tremendous power. The notion can seem daunting, but I’m hoping to demystify javascript closures for you, and maybe even make them seem friendly and useful!

Before we get to the technicals, I'd like to share a quick analogy that I appreciate. I like think of closures as programming DNA; a replicating material that carries genetic information. Closures can give birth to new values or data structures, and can modify existing ones.

There are two important concepts in Javascript, that when combined, pave the way for closures:

Variables that are local to a function are recreated everytime the function is called.

Functions can be treated as values.

Let's take a look at an example:

function whatsTheValue(x) { var theLocalVariable = x; return function() { return theLocalVariable; }; } /* theLocalVariable is defined within whatsTheValue(), and each time whatsTheValue() is called, theLocalVariable is redefined in the new instance */ var instance1 = whatsTheValue('hello'); var instance2 = whatsTheValue('world'); /* whatsTheValue('hello') and whatsTheValue('world') are assigned to separate values */ console.log( instance1() ); //the output is 'hello' console.log( instance2() ); //the output is 'world'

A function that “encloses” specific local variables is aptly named a closure. Let's look at a slightly more advanced example of this. In the following case, we'll enclose an initial value within a function, which we'll then later use against a second parameter:

function subtract(amountToSubtract) { return function(amountToBeSubtractedAgainst) { return amountToBeSubtractedAgainst - amountToSubtract; }; } /* we define our closure, which creates a function that subtracts by the amount amountToSubtract */ var subtractTen = subtract(10); /* we create a specific instance of subtract with it's own name, that when invoked, subtracts 10 from the value it's later given */ console.log( subtractTen(100) ); //the output is 90 var subtractFive = subtract(5); /* we create another specific instance of subtract with it's own name, that when invoked, subtracts 5 from the value it's later given */ console.log( subtractFive(100) ); //the output is 95

If the above examples don't quite make sense, it's ok. I suggest reviewing them and playing around with your own closure examples by modifying the above code. Once you're comfortable, let's take it up another notch. Closures can also:

Update their internal variables.

Contain multiple functions.

Let's explore these two concepts:

function createClosureWithID (initialID) { var closureID = initialID; /* our local variable is defined */ return { getClosureID: function () { /* we define a function within the closure that returns closureID */ return closureID; }, setClosureID: function (newID) { /* we define a second function within the closure that permanently updates this specific instance closureID */ closureID = newID; } } } var closureInstance = createClosureWithID(1); /* closureInstance has been created and its closureID is 1 */ closureInstance.getClosureID(); //this returns 1 closureInstance.setClosureID(2); //firstClosureInstance's closureID is now 2 closureInstance.getClosureID(); //this returns 2

You may be starting to see how closures can play such powerful role in Javascript's object oriented programming. If not, don't sweat it - we're going to continue to cover it in the next blog about closures: How do Javascript closures work? Intermediate concepts...

Disclaimer: This article won’t help you learn React.js or Node.js - it is basic instruction on how to get your React/Node project from npm start (?) on your local machine to an Ubuntu web server.

I’ve fallen head over heels for the React.js framework. When it came time to put my first React/Node project on a web server, I realized that it was a little different from the traditional LAMP setup that I was used to. However - using an Ubuntu web server, Nginx, and Node.js - I’ll show you a simple method to get your project up and running online.

There are two prerequisites:

You have an Ubuntu 16.04 server, configured with a non-root user with sudo privileges.

You have a React.js project that is readily available to place onto your Ubuntu server.

Let’s get started!

First, ssh into your server. Then, let’s make sure your server’s packages and dependencies are up to date using the following:

$ apt-get update

Node.js ...?

The next thing we want to do is install Node.js from the NodeSource package archives. So, let’s make sure we’re in the home directory of our server using the following:

$ cd ~

Let’s curl retrieve the NodeSource PPA, which will give us the installation script we need to install Node.js, using the following:

$ curl -sL https://deb.nodesource.com/setup_6.x -o nodesource_setup.sh

This created the file nodesource_setup.sh in our current directory. We want to run this script under sudo using the following:

$ sudo bash nodesource_setup.sh

Boom - now that we’ve run the setup script, we can install the Node.js package and the necessary dependencies like so:

$ sudo apt-get install nodejs build-essential

One thing to note - since we installed from the NodeSource PPA, the Node.js executable is called nodejs as opposed to node

We’ve got Node.js set up on the server - now what?

Let's get your project on to the server!

You can put your project in the same directory that you're already in (the top level directory accessed via $ cd ~). I use Github to host my projects, so let's make sure git is available for use:

$ sudo apt-get -y install git

Now clone your project onto your server like so:

$ git clone {your project's clone url}

Assuming you have some dependencies that go along with your project, you'll want to make sure those are readily available. Let's cd into our project, then grab those dependencies:

$ cd {your project folder} $ npm install

Now, normally on your local machine, you would start your app using npm start (?) then your server would tell you that your app is being served at http://localhost:8080/. This, however, poses a couple problems on a web server:

The server doesn't know that it should be sending visitors to the 8080 port by default - it doesn't know that your application lives there... yet.

Your app, which we started with npm start, will eventually die when your ssh session to your server times out - thus killing the process that is serving your app.

Keeping the process alive

Let's install a process manager to make sure that once we start our app, it is managed and maintained. We're going to use pm2, and install it using the following:

$ sudo npm install -g pm2

Now you can have pm2 manage processes globally, however, it can't directly run the npm start command.

What we'll do is wrap the npm start command in a bash script so that pm2 can run it for us. This script file should be in the same directory as your project folder (the top level directory accessed via $ cd ~). Let's create the bash script file like so:

$ touch start.sh

Now, we need to access the start.sh script so that we can modify it:

$ nano start.sh

Then, let's add the following three lines of code to this file:

#!/bin/bash cd {your project folder name} npm start

What the above script does for us is allow pm2 to execute following:

Enter into your project's directory

Execute npm start

Now we can feed this script to pm2, and it can start and monitor our application for us using npm start just like we would manually.

To test this out, execute the following command:

$ pm2 start start.sh

You won't get the same output that you're used to seeing once you execute npm start. What you'll see is that pm2 let's you know that it is monitoring the process, like so:

For detailed documentation on the full monitoring capabilities of pm2 please please please see their docs

Defaulting public traffic to 8080

Now that we have pm2 watching our back on the server for us, the only thing left to do is to tell the server that we want it to default visitors to the 8080 port that our application is being served on.

For that, we will use nginx as a reverse proxy. First, let's install nginx by executing the following:

$ sudo apt-get install nginx

Ok sweet - we've got nginx on our server, now we need to tell it what to do for us. Let's open up the default server block configuration like so:

$ sudo nano /etc/nginx/sites-available/default

Next, you'll want to delete everything inside this file. Then, you'll want to add the following:

This configures our server to default it's public root path to our app, which is being served at 8080. Next, in order for these changes to take place, we'll need to restart nginx like so:

sudo systemctl restart nginx

Now, when accessing your server's ip address or base URL, you should see your app being served as the index content.