CrowdHailer

Grunt is nothing but a neat way to package tasks which are added as plugins. Essentially grunt has no value without the plugins that its community has generated. With these grunt is wonderful.

So with that here is a whirlwind intro to some of the grunt tasks I have made use of.

Concatenation contrib-concat

With this plugin you can automatically gather JavaScript/CSS saved in several locations and combine them to one file. This is best practise for performance by reducing HTTP request and speeding page load time. 

Minification contrib-uglify, css-min

There are plugins to minify JS, CSS and HTML. This decreases the sizes of assets by removing comments, white space and several other things that human readers find very useful. This reduced size of files to be served is also helpful in speeding up a page load time.

Watch contrib-watch

Automate the automated tasks, this task watches for changes to files, upon which it will run other grunt tasks. Save any changes in your js files and a new minified version will be ready almost instantaneously

Linting jslint 

Existing for JavaScript and CSS these task will check the quality of code you are writing, The value of jslint at catching mistakes is really huge.

Some More interesting task

The above are a powerful toolkit on there own, this next selection are more for the  curious

load-grunt-tasks

Efficiency or Laziness, whichever it is, it becomes contagious once grunt is running. This task saves you from having to individually add every grunt plugin to your grunt file. And will instead load every node package that is prefixed with 'grunt-'

time-grunt

This will record the time that each task takes to run. If a task is build of several tasks it will tell you the percentage time on each. So far this is more interesting than useful. It might provide early warning if you are minifying lots of images that you have too many as the time creeps up.

grunt-newer

This task provides can prefix any other task and will only run that task again if the files it is acting upon have changed since the last time the task was run. A useful way of speeding things up.

localtunnel-me

Not a grunt task but a great addition to the development workflow. Previously I was pushing a version of my project to heroku to show it to other people. This meant that my project was littered with commits such as 'test this new thing on phone'. Now this is all replaced with localtunnel. Grunt has several plugins that allow you to spin up a server with your project. with local tunnel you can take a locally running server and with no knowledge of how make it available the world over with one command

$ lt --port 8000

Web Components

This is a conglomeration of emergent specifications that will allow pages to be divided more along the lines of widgets/sections. This is instead of separation into just three functional pieces namely content(HTML), style(CSS) and behaviour(JavaScript). The four new web standards for web components are

Custom Elements

HTML Imports

Templates

Shadow DOM

Browser support is variable and progress can be checked here. They can be experimented with now using a polyfill library called polymer. A decent introduction can be found here.

Functional Reactive Programming (FRP)

I first came across this in the excellent JavaScript library bacon.js. There are other implementations such as rx.js. FRP is a structure to handle events and data manipulation. It aims to provide a consistent language for manipulating event stream that have much in common with collection operations such as map, reduce and filer. The analogue being that a stream of clicks can be considered an array in time of click. How helpful that visualisation is I am unsure off but the code is definitely a wonderful step up the abstraction ladder. To have an overview I would definitely recommend this walk through of implementing snake

Microservices

Fundamentally this is about breaking up a well organised large application into several smaller applications that run in concert to create on application agglomeration to a user. There are several benefits to this approach, each can be written in the language most suitable to its task, they can easily be rewritten when the business environment changes. The downsides are mostly the added complexity with deploying many services but this can be mitigated with automation and making use of services such as heroku. An excellent introduction is here.

Single Responsibility

The Single Responsibility Principle (SRP) is a cornerstone of good code design but there is more to it that the simplicity of its statement gives away. It took me a while to discover what I see as a key feature of single responsibility and that is a concept of scale.

In a restaurant the chef has the single responsibility to prepare food while the waiter has the single responsibility to deliver food. However into this mix we can add a tin opener that has the single responsibility to open tins. These are however not three equal partners. It is evident that the chef can use the tin opener without having two responsibilities, namely preparing food and operating a tin opener. Operating a tin opener is part of the preparing food responsibility.

I have been building an application over the last few weeks with a greater scope than any I have done before. Following the mantra of fat model I have always aimed to move logic into the models. I not have the situation on obese models and need to break them up further.

This post has a good overview of the problem and what an active record model should not do.

From code climate this one has some good advice on where to start breaking up these models.

One assertion that I have read a few times recently is that authorization logic should remain in the controller. I have followed this but as I have introduced admin and content owners and public pages this has caused my controllers to grow more than I am happy with. This is a good overview of using exceptions to manage complexity but I would still be intrigued to read further arguments on why this logic can't be pushed to the controller.

Next the Javascript

Node Webkit. That sounds cool, this guys blog also has a whole bunch of stuff. Onsen sounds fun too. Polyfills as a service, why the fresh not, from the financial times guys.

This article is not an exciting new tech but just quite a nice overview of several JavaScript hacks.

Python time

I have a growing urge to program in python and this post is a good example of some of the stuff the python community is leading the way in. So also needed is a MEGA-tutorial. Check.

Finally I have no Idea what the bejingo

Well I have started looking again and I have changed my mind drastically. I decided I needed to look again after watch this excellent talk from Paul Irish. Yeoman fulfils one role fantastically and that role is as a demonstrator. When pushing yourself to learn new techniques I find myself looking at a lot of other peoples code. This is great but most of the time you find yourself looking at a finished product. This means that the implementation details such as build steps are hidden around all the actual code. Trying to learn about simple grunt tasks by looking at the JQuery repository will feel overwhelming. Even though you might be aware that you should only need to look through their grunt file you are never sure if something they are running in the gruntfile is dependent on a particular structure elsewhere in the repo. This is were Yeoman steps in, Yeoman's scaffolds are all the unglamorous bits of a project with all the excitement of the application stripped away.

Being a scaffolding system it is opinionated, many of the individual scaffolds are not overly flexible. Being a relative newcomer however studying these scaffolds has ended up me arriving at the realisation that the workflows promoted in these proto applications are superior to what I was managing on my own. I now make frequent use of Yeomans webapp scaffold.

Getting started with yeoman.

Read about grunt, bower, node & npm. You don't need a deep understanding of any one of these. However understanding what each is for can be useful when working out how the whole thing fits together. 

With that out of the way lets get started. Assuming that npm is up and running yeoman is installed using

npm install -g yo

yo then becomes a global command. Running yo brings up a helpful list of options which will walk you through things like finding and installing a generator (project template) as well as building a new project from a template.

All yeoman generators are separate packages and can be installed directly with npm. To install the webapp generator and use it.

npm install -g generator-webapp

mkdir newProject

cd newProject

yo webapp

Next steps

Open the project and have a look around. Even if you never use a generator in one of your own projects you will learn a lot.

Some other scaffolds I have found interesting

Socketio

Zf5

mobile

gulp-webapp

Principles

A globally accessible object is responsible for passing a messages that is triggered on one channel to all subscribers of that channel. The other objects in the software system are only activated by broadcasts on the channel they listen to and can only communicate the results they generate by broadcasts. Importantly they will have no idea who is listening. 

As an example a completed ajax callback may take the returned data and broadcast that new data is available. A separate object can then write the data to the screen. The original ajax object does not know or care what happened to the data it retrieved. 

Advantages

The events are universal and can easily be re-purposed. In the example above if data is loaded by another means, perhaps from local storage, the same event can be fired and with no further changes it will be written to the screen. Tests and logging can be added implemented on the same events. 

Disadvantages

If an event is lost, for example objects are broadcasting on channel while the screen writer is listen on a different on, are lost permanently. There is no true/false return for successful completion of an action. In reality this is normally a good thing as waiting for those confirmations particularly in asynchronous code leads to alot of code spaghetti. However when testing it was something that I learnt to check first when tests where failing. 

Implementation

Control object

The singleton pattern is very useful for this as it is imperative that all calls to broadcast reach the same object. The following code creates a tower instance the first time it is called and in all subsequent calls returns the first created instance.

(function (global) {

  var instance;

  function init(){

    instance = // code goes here ...

    return instance

  }

  global.Belfry = {

    getTower: function(){

      if (!instance) {

        instance = init();

      }

      return instance;

    }

  };

}(this));

Subscribing

Each channel is an array on a channels object. When subscribing a function to a channel the function is added to the corresponding array. 

function subscribe(topic, reaction){

    var channel = channels[topic] = channels[topic] || [];

    channel.push = reaction;

}

Publishing

When a broadcast is made each function in that channels array is called with the content that is to be added to the broadcast.

function publish(topic, content){

    _.each(function(action){

        action(content, topic);

    })(channels[topic] || {});

}

And that is in essence the components of a pubsub. Checkout my library belfry to see a full implementation. There are several embellishments, such as the ability to unsubscribe. The functionality is also curried as this allows broadcast functions for individual channels to be generated which can then be passed around. This reduces the chances of broadcasting on the wrong channel. 

Karma runs on node.js and is easiest installed using the node package manager (npm). If you have not got either of these installed checkout my post on getting started with node and npm.

Karma is a test runner and requires a test framework to work with. Supported frameworks are jasmine, qunit and mocha. A discussion of the differences between these is beyond this post. In summary I chose Jasmine as it is the most complete without add ons.

Karma also requires a browser to run the tests and a plugin to handle that browser. Supported browsers are Chrome, ChromeCanary, FireFox, Safari, PhantomJS, Opera and InternetExplorer.

Getting started the first step is to initialise a node project then add the test dependencies to package.json.

"devDependencies": {

    "karma-jasmine": "~0.1.3",

    "karma-phantomjs-launcher": "~0.1.1",

    "karma": "~0.10.6",

    "karma-spec-reporter": "0.0.12",

  }

Running 'npm install' will install all of the packages locally to node. At this point to run any of the karma commands you will be required to enter the whole path, to run the tests for example is likely to be.

$ ./node_modules/karma/bin/karma start

This is tedious and fortunately karma also has a tool to add these commands to your path. To add the command karma to the terminal the package karma-cli needs to be installed globally. 

$ npm install -g karma-cli

$ karma start  // same as long command above

The configuration settings are kept in karma.conf.js. Here you set how you would like to format the output, whether to run continuously and what files to load. There are a lot of whys to set up karma, this is my preference.

module.exports = function(config) {   config.set({     basePath: '',     frameworks: ['jasmine'],     files: [       'bower_components/path/to/dependency.js',       'src/*.js',       'spec/**_spec.js',     ],     reporters: ['spec'],     port: 9876,     colors: true,     logLevel: config.LOG_INFO,     autoWatch: false,     browsers: ['PhantomJS'],     captureTimeout: 60000,     singleRun: true   }); };

An important thing to note is to remember to load up all your dependencies, also when using wildcards the files will be added in alphabetical order. When the order is important then that will need to be declared specifically. It took me a while to spot that one as the first three projects I did the correct order was alphabetical order

I always use the test suite as a single run through. As the test suites I have written so far take less than 1 sec I have always been happy to explicitly run them. This also works well with continuous integration.  

Finally the 'spec' reporter is not included by default and needs to be added through node

$ npm install karma-spec-reporter --save-dev

All of the examples will be in JavaScript as it is very easy to pass around functions. in fact this is one of the reasons I like JavaScript.

Currying

I have found it best described as a way of incrementally declaring arguments. Currying relies on the fact that functions can return functions. First some code for a function to add two numbers, in what could be considered as the classic way.

var add = function(a, b){

  return a + b;

};

add(3, 5)

=> 8

This code is very simple and appears to have no downsides so why would you change it. The hidden downside is that it requires you to provide both 'a' & 'b' at the same time. Contrast with the curried function below

var add = function(a){

  return function(b){

    return a + b;

  };

};

var add3 = add(3);

add3(5);

In this code first the value of 3 is passed to the add function. This creates a brand new function that will always add 3 to which the value of 5 is passed. This may not seam that revolutionary but can be very useful with operations that are often used. e.g calculating a tax percentage at always 17.5%.

Function composition

Composition is the process of 'piping' the output of one function 'f' to the input of a second 'g' before the first input is present. In the process creating a third function 'h' which is the equivalent of sequentially applying 'f' then 'g'.

h(x) = g(f(x)) 

This is all very theoretical and will be better explained by an example. Imagine we want to add 3 to a number and then double the result.

First lets introduce a curried multiply function similar to the curried add function. 

var multiply = function(a){

  return function(b){

    return a * b;

  };

};

Then we create the components of add3 and double

var add3 = add(3);

var double = multiply(2);

We can then pass both of these to our compose function and just use the resulting function.

var operation = compose(double, add3)

operation(1)

=> 8

operation(3)

=> 12

The benefit of this is it allows us to set up very sophisticated operations, all describe in one variable, before any of the data has arrived. The 'dumb' data is then manipulated in very few steps and intermediate often uninteresting variables are skipped over.

Implementation

Function composition is simple in theory but it implementation can be more tricky, due to complexity in preserving the composed function's context (what the 'this' object refers to when executing the function). Solving this requires a good understanding of JavaScripts bind, call and apply. Ideally implementation details would be taken care of and the functionality described above would be available in a Library.

To this end I have been developing 'cumin.js' a functional utilities library to experiment with these techniques. To build a solid base I have started by reflecting much of the functions available in underscore.js. The single rule is that the data should come last. This is explained in this great talk 'Hey Underscore, your doing it wrong' from Brian Lonsdorf.

Getting Bower

Bower itself is a node package and so first you must ensure that your system has node and npm. You can see my blog post on starting with these. Now you have these on your system you can start with bower. Install Bower globally with the following command.

npm install -g bower

Bower packages

Packages can be installed directly and bower  will save the them to the bower_components directory by default.

bower install <package-name> 

Bower can also use a bower.json file to install packages. There is an init function that will interactively create one, providing helpful defaults for name and version.

bower init

If you are only creating a private project this file is most useful for saving dependencies. Packages can be manually added to the json file. The install command also includes a switch to automatically save installed packages

bower install <package-name> --save

In the future running bower install will automatically install all dependencies mentioned in the json file.  

There are a huge number of available packages, from simple utilities to full frameworks such as ionic.

Registering a package

Running off git it is fantastically easy to create a bower package. First make sure that the bower.json file is created and contains information such as name, author and keywords. Then the package can be registered using

bower register <name> <url>

The name is the name the package will be registered under and has to be unique. The url is to a public git repository where the package is located. And that's it.

Built on top of git creating a new release is as simple as tagging a commit in your repo and pushing it to the remote storage.

Using Bower

Registering a package is so simple that I have found it an easy way to manage code even within projects. Recently I have been writing an application to manipulate SVG images. During development I have created several useful utilities that I wanted to factor out of the main code base. Instead of storing them in separate files, but still in the main repository, and combining them using a grunt task. I have added them as bower packages. This makes them far more accessible to future projects. Most importantly this has made them available to others and helps to grow opensource code bases. Something that I can feel good about even if they may not have been used by anyone. My bower packages so far are:-

Cumin, utilities library favouring curried functions and functional operations

Belfry, a simple publish subscribe library returning curried functions to work with cumin

SoVeryGroovy, convenient API for generating SVG components

Installation

This installation guide has been tested for myself using Linux Mint and should probably work on Ubuntu. The core ppa contains a node version however the ppa maintained by Chris Lea is bundled with npm and provides a very convenient choice for downloading. First add his ppa and the fetch nodejs.

$ sudo add-apt-repository ppa:chris-lea/node.js $ sudo apt-get update $ sudo apt-get install nodejs

This should have added the command 'node' and 'npm' to your terminal. running 'node' will enter the read evaluate print loop (repl) for node. 

Starting a node project

All details of a node project are kept in 'package.json'. This file contains information such as name, author, version number and dependencies. Npm provides an init method which runs a helper to generate the package file. Starting a project looks like this

$ mkdir node-project $ cd node-project $ npm init

To install a node package to the project use install with the save switch or optionally the save-dev switch for packages that are only required when testing/developing.

$ npm install <name> --save $ npm install <name> --save-dev

Some packages will need to be installed globally. This is likely the case for ones that are adding a utility to your terminal. Such as 'grunt', 'bower', etc. This is achieved with the global switch

$ sudo npm install -g <name>

Sharing a node project

Packages installed locally will, by default, be saved in the directory node_modules. These dependencies can be populated by npm from package.json and so do not belong in version control. Ensure that you add 'node_modules' to your .gitignore file before committing.

When cloning a node project you will need to add the dependencies. Running npm install with no arguments will fetch dependencies specified in the package file and add them locally.

First up I took a week off, this I would recommend to anyone following. It helps to remember that a Saturday needn't be spent watching a rails tutorial. So I saw some friends in France and Switzerland and that was all very nice. I discussed the merits of Test Driven Development and Pair Programming with a friend of mine in academia and in return he introduced me to 2048. If you haven't found this yet its probably best to stay that way. 

Back to work and the first thing I wanted to tackle was consolidating my knowledge from the course. There were tasks I did not manage to complete as well as many helpful resources that I want to go over. For an idea of just how many useful resources we collected over the 12 weeks check out this repository were we have consolidated some.

Next up was cleaning my Github account. (yes I know this should have happened as I went along). This was the only thing on my list that took less time that expected. I had been trying to write sensible README's as I went along and in most cases a quick overview is all that is required.

Of less importance but most fun was exploring brand new things. Continuous integration is something we heard and I have enjoyed experimented with Codeship. Also automated code review for which I have experimented with Codeclimate. These are both really easy to integrate with Github and I was able to set-up both of these and complete a web based fizzbuzz kata in under 30 mins. To start serious TDD with JavaScript I have explored using Bower, Karma, Grunt, Yeoman and Jasmine-Jquery. All of which are worthy of there own blog post.

With all of these to do's now under control, about three weeks after graduating, I can start to make meaning full progress on finishing off my London Layout App. Thankfully with my new expertise what were serious problems before Makers I can now easily overcome. 

First I needed pip, a package management system for software written in python. Second Virtualenv, which is used to create isolated Python environments. This addresses the same role as Ruby Gemfiles and Bundler. These two are installed with apt-get.

    $ sudo apt-get install python-pip     $ sudo apt-get install python-virtualenv

Also recommended was the nattily named virtualenvwrapper. This can be installed in the same way but requires some further setup. To set the location of project and the python environments I had to add the following to my bash profile. 

    export WORKON_HOME=$HOME/.virtualenvs     export PROJECT_HOME=$HOME/Devel     source /usr/local/bin/virtualenvwrapper.sh

With this setup the project can now be started making full use of the tools installed above. Amazingly these tools mean that it is possible to have a project live on Heroku with only 10 commands from the terminal. I have split these into two groups of 5. The first being

$ mkproject hello

$ pip install Flask gunicorn

$ pip freeze > requirements.txt

$ touch hello.py 

$ echo 'web: gunicorn hello:app' > Procfile

So what happened here? mkproject is a command from virtualenvwrapper, it creates an environment called hello and and project called hello. Then it switches to the project directory and starts the created environment. Both Flask and gunicorn are python packages that we install to the enviroment. pip freeze lists all packages that are currently in use, saving them to requirements.txt allows Heroku to install the same ones. we then create an application file called hello.py and a Procfile to give instructions on starting the application.

At this point we can write an application in the hello.py file. For a quick example I just used the sample application for Heroku for getting started.

So at this point we have a working application that can be run locally with foreman. Next is to push this to Heroku. These steps should be familiar if you have previously used Heroku however I have listed them below for completeness. 

$ git init

$ git add .

$ git commit -m 'init'

$ heroku create

$ git push heroku master

At this point your application is live on Heroku. Running a final 'Heroku open' will open the application in your default browser to prove the point.