Sid Azad

Web apps/sites today don’t look the way they used to a decade ago. In fact, they look very different from what they looked 5 years ago. With javascript frameworks such as AngularJS, ReactJS, Backbone/Ember as well as server frameworks such as Django, Flask, Node, Rails the way you test your web site/app has changed as well. 

The purpose of this post is to:

1. Think about a decent organization of the various kinds of tests for your web app/site

2. Think about the pros and cons of using the various combinations of tests

3. Present one (of several) ways to organize various tests for a modern web-app

The diagram posted on top shows a typical simple web-app. Our goal is to test the various parts of the above diagram, either as standalone units or as a whole (or a combination thereof) with the least amount of redundant testing.

There are various different tests you could have in such a system. I will consider a system using:

1. AngularJS + jQuery front end

2. Django Back end

But the analysis here should be easily applicable to a different front end (such as ReactJS or EmberJS) or back end (such as Rails or Flask or Node).

The different kinds of tests in such as system could be:

1. AngularJS Unit tests

2. JavaScript module unit tests

3. Django unit tests

4. Unit tests for standalone Python Libraries (used in the server)

5. End to End tests (using Selenium for example)

We need to make a decision whether or not to support each of these kinds of tests, how much weight to give to each and how to measure the coverage for each of these. 

Let’s think about what each of these tests are testing, and what they are not testing. 

1. AngularJS Unit tests - These tests are testing your Angular Components such as directives, controllers, services in a standalone fashion (as if these components exist outside of the overall system). In short, they are testing some but not  all components that run inside the browser in the above diagram. Also remember that there may be pages that aren’t completely or perfectly componentized such as those using 3rd party libraries such as jQuery plugins.

2. JavaScript Module unit tests - Assuming that you have modularized most of your JavaScript code (which is highly recommended), you can test these modules as standalone components. These tests fall in the same category as (1) that is AngularJS Unit tests as by testing these you will be testing certain parts of the system that run in the browser, in a standalone fashion. 3. Django Unit Tests -  Django provides a unit test runner that can invoke Django views in the same way as a browser client would. There are actually 2 ways to use these tests. 

3.1 You could mock out your db access part (for example if usually your view  retrieves data from a DB, you could substitute with a mock api that either returns hard-coded data or data from a file system. In this case, you will not be testing the circuit labeled (3) in the diagram above. 3.2 You could test the full circuit against a test DB such that the circuit           labeled (3) is also tested via your Django unit tests.

4. Unit tests for standalone python libraries - Surely your Django views must be using other standalone Python libraries and you should have separate unit tests for these components, especially since these components could very well be used outside of the Django environment by other (possibly non-web) applications.

5. End to End tests - Finally, web-apps these days are commonly relying on test frameworks such as selenium to invoke end to end requests in a headless or browser based environment to test the full circuit. 

These tests are great as they test the entire system but come with a few problems:

- In my experience modern rich UIs aren’t as easy to test with Selenium as the headless browser isn’t always up to date with features you may be using - These tests are time-consuming, to design as well as to run. To design you  need to record user requests via UI interaction. Running these tests invokes the full circuit (from browser to db and back) and need all parts of the             system to be up and running at all times) - There is some level of redundancy here since in each test you would end up testing the circuits 1, 2 and 3 in the diagrams above and there may not be a huge benefit to that - the argument being that if the data packaging, serialization to JSON etc. and deserialization is working for Page 1 then it probably doesn’t need to be tested for Page 2. 

It should be clear that to test our system in a practical and thorough way, we need to use a combination of these tests. Using only End to End tests is not a feasible solution, as it ends up being very time consuming to write as well as to run, and that could server as a deterrent to writing more tests and running them often.

The big question is, what is the right combination of these tests. There could be several opinions and answers to that, and each system and team is different in it’s needs and environment. One way to combine these tests is presented here without claiming that this is the best or perfect way to do so:

A Possible Organization

The summary of the method is to use all of these tests together with different weights given to each. The idea is to test most of the core logic using various unit tests and have only a few end to end tests. To elaborate:

1. Use AngularJS and JavaScript unit tests (1 and 2) using a tool such as Karma/Jasmine/Mocha and measure coverage using a coverage reporter. Test each JavaScript module.

2. Write Django unit tests for each view and test as many code paths as you can and measure coverage using coverage.py . I recommend using a test database as opposed to mocking out the DAOs. This is especially better if you are using something like SQL Alchemy or Mongo QL since you would then end up testing the logic of your DAOs as well. This obviously is not an absolute rule and there could and would be exceptions.

3. Unit test your standalone Python (or Java for that matter) libraries.

4. End to End tests: Use a tool such as Selenium to write end to end tests BUT use this in a broad way. The goal is to ensure that:

- The overall circuit is working - The pages that aren’t well componentized are tested

For example, you could have tests for:

- Login, Logout (Authentication) - Have at least 1 end to end test for each page.  - For example if you have a lot of search pages then invoke the search for each page, but don’t test ALL the possible combinations of input search fields as the Django Unit tests should be testing those.  - If you have updates and insert then it is recommended to have a test for the update as well as the insert testing from the point the users fills a form and presses a button to getting a response back and verifying that the data was actually updated/inserted.

Remember that end to end tests are very different from unit tests for the front end as unit tests (whether AngularJS or JavaScript) are only testing a single or a few components rather than the interaction of various components (jQuery, AngularJS) with each other and with the DOM.

I have not covered event or pub/sub related tests such as for Celery, RabbitMQ related components but you should definitely have some end to end and many unit tests for the same. 

The overarching summary of this post is:

1. Write many unit tests on each tier of the stack and try to make them run fast 2. Write some end to end tests since they slow the system down, relying on the fact that you have thoroughly tested the core logic of the system via unit tests. I hope this provides a line of thinking towards testing a modern web-app. This isn’t meant to be definitive or the only way to organize your test stack, but is one method. 

Comments are welcome.

References:

1. http://stackoverflow.com/questions/182325/why-are-functional-tests-not-enough-what-do-unit-tests-offer?rq=1

I just read this article describing Stephen Hawking's interview in which he said "The development of full artificial intelligence could spell the end of the human race. Not too long ago Elon Musk called AI "our biggest existential threat". http://www.cnbc.com/id/102121127

My first thought was to dismiss these statements as paranoia but then just for the fun of it I started thinking about how AI could possibly pose a danger to humanity. One way to think about this is a thought experiment. Let's try to imagine an AI that can be a threat to humanity. Imagine the following scenario:

A very smart person (or group of people) with large amounts of resources at their disposal build a vast AI machine.

The machine has a large amount of computing/processing power

It has access to unlimited amounts of electricity or another source of power (or so much that it can be considered unlimited)

It has enough backup power that it can go on for say a 100 years at full capacity even if the source of electricity is disconnected

It has 360 degree vision recognition capacity so much so that it can identify any person on the planet

It has aerial (flying) capabilities

It has various kinds of missiles and warheads on board as well as precise targeting scanners

It has voice recognition such that it can communicate in any language (based on all the corpora available since the history of the earth)

It has a knowledge representation system which allows it to aggregate data from it's environment and "learn" from it

With these specs in mind let's think about what the people who designed the machine intend to do with it. There could be at least 3 scenarios:

The intention is not evil and the team has created this machine to guard humanity against possible aliens (or guard a country against an invasion)

The intention is evil and the team wants to use this machine to exert power over others (take over a country or the world)

The team is comprised of nihilistic people who don't care for the world and want to unleash the machine on the world without caring about the consequences.

In the above 2 cases the AI has been programmed with a particular "goal" or "intent" (let's use the term "intent" to give our discussion some personality). In the 3rd case, there is no clear intent then to cause chaos.

It seems to me that in the first 2 cases the AI is under control of the creators since there is an intent provided to the AI. As long as the creators have been smart enough to program the AI in such a way that it's learning algorithms never override it's original intent, the AI may deviate in behavior from the creators vision, but it will not deviate in the final result.

The 3rd case is more problematic since the creators may have designed the AI in such a way that it can randomly (or seemingly randomly) modify it's intent and result in chaos.

Now let's consider the following scenario.

The AI has been built with the best of intentions, let's say (1), of defending a country or the earth

Now tragically all the creators die in a plane crash and the AI is on its own

The code for the AI was not open source, so nobody knows how the algorithm/AI functions

There is a bug in the AI due to which it starts perceiving friendly objects as threats and starts shooting at them

How would we fix this? Remember that the AI is extremely powerful and can see what's coming at it due to vision recognition and endless computing power and can face any threats to itself. We may have at this point some alternatives:

Try to find a loophole in the AI's algorithm and trap it - the problem is that we don't have access to the source code

Build another, more powerful AI and hope that the AI that has gone wild does not detect this via energy signatures and come to destroy it

Move to another planet (or colonize the moon)

So what seemed like paranoia begins to become more fathomable with some assumptions. Given Moore's law and the amount of computing power and memory already available to us, as well as mega and giga-batteries, some of the assumptions don't seem that unlikely.

But is this really Artificial Intelligence? 

Humans programmed this machine

They made rules via which the machine learns (granted the rules, such as neural nets and SVMs) are so complex that human beings can't understand/analyze them after a certain point

Humans are responsible for programming the intent of the machine. It can't be so hard to ensure that the machine can NEVER divert from the original intent and have a few fallback options.

So in conclusion, can machines destroy humanity? Maybe, if we build a machine capable of inflicting such destruction and if a lot of other things go wrong (software bugs, lack of fallback options). It is in the programmer's and policy makers hands as to how much control he/she gives to the machine. A machine follows instructions. A well designed machine will work as expected. A machine with a clear and consistent policy structure will adhere to it, even if it has a learning algorithm.

I have been writing Single Page Apps and websites using frameworks such as Backbone, AngularJS for some time now. Often I come across websites that are very similar to each other.  For example, most websites have a landing page which have a Header/Center/Footer layout. The landing page usually has a widget to login and sign-up, via which the user can login and get to the Dashboard. Let's call this the Login-Dashboard pattern. Another aspect of the web site/app development process is the framework used. I could implement a website such as described in the Login-Dashboard pattern above using a framework such as AngularJS or Backbone or Ember or plain old JavaScript and/or jQuery.  Given the 2 main factors above it would be nice to have a tool which would satisfy the following requirements:

Having described the website once provide the ability to generate the website using different frameworks/libraries such as AngularJS or Backbone (or something else). Why should we have to write the whole website again to switch the underlying framework.

Have the ability to describe the website/app in terms of high level patterns (such as the Login-Dashboard pattern mentioned above). After all most websites have many common patterns.

The 2 requirements above feed into each other as the better the generic patterns in terms of which the website or app is described the easier it would be to standardize and generate bindings for various frameworks.

Keeping the above in mind I started a project called 'Tersity' which allows describing a website in Python in a 'terse' way (hence the name). Currently I have provided AngularJS  bindings some common Layouts (such as Header-Center-Footer) and a couple of common patterns. Abstractions for UI Elements such as Pages, Layouts, Sections, Rows and Columns have been provided as Python classes.

If you are using any client side single page application frameworks such as AngularJS, Backbone, Ember.js, then you have dealt with client side routing of URLs and are aware of some of the quirks that come with it. Initially to accommodate client side routing, hashbang urls were used, that is URLs with domain.com/#!/something/somethingelse. Anything after the #! was left for the client framework to interpret.

Now hashbang routes aren't the most natural. Even twitter has eliminated them. AngularJS also provides a way to eliminate hashbangs. $location service has a method html5mode() which when set to true will make the URLs lose the hashbang.But there is something important that you need to be aware of.

Impact On The Server Side

Let's consider a URL: http://angularjstutor.com/coursedes. This URL is from my website which is in AngularJS. When you are using hasbangs, this URL looks like http://www.angularjstutor.com/#!/coursedes. This means that in hashbang mode the server sees the URL "/" when the page is request. Contrast that with non-hashbang mode (html5mode set to true) where the server would see the URL /coursedes. This means that if you don't have a handler for the URL /coursedes, and you have switched from HTML5 mode false to true, your application will break.

Manifestation Of The Problem

This problem will not manifest when you are clicking links in your website, since you have already loaded AngularJS by that point. AngularJS will see the /coursedes and handle it on the client side. It manifests when you go directly to the URL or if you are on the http://angularjstutor.com/coursedes page and you hit the browser Refresh button. Then a fresh request is sent by the browser to the server and that cannot be intercepted by AngularJS because it's a new invocation by the browser.

Solution

The solution is to either have a catch-all handler or do a URL rewrite on your server to handle all URLs. But what should this handler return? It should simply return the same response that is returned in your base URL request. Usually this file is named index.html. In any case, it should be a file that contains the AngularJS libraries to be loaded. Once you do that, AngularJS routing will take over and see that the URL has a /coursedes and will invoke the appropriate routes.

Performace

If you are concerned that going back to the server and retrieving the index page again, with all the javascript and css that comes with it is an unnecessary performance hit then it is important to keep in mind the following:

This performance hit would occur whether you use HTML5 or Hashbang routes since in both cases you would make 2 trips to the server. The first to retrieve the index page and the second when the AngularJS routing kicks in and retrieves the template from the server side (Of course you can avoid the second one by keeping the template on the client from the git-go)

The second hit can be avoided if you aren't using a client side framework since in that case you can retrieve the whole page in one shot, as the server would be well aware of what the complete route means.

But in most cases a second trip to the server may not be a huge concern.

Video

This is a rather subtle issue and a bit difficult to explain in a text post so I have made a video which you can view here: Demystifying AngularJS Routing - HTML5 and Hashbang modes

Resources

Here are some of the resources used in this post:

Code for the www.angularjstutor.com site

AngularJS Tutor

AngularJS UI Router

AngularJS is a powerful JavaScript framework which has turned Front-end development on it's head and made it much less tedious and much more efficient than it used to be. IMHO it is the future of Front End Development.

The novel approach to Front-end development taken by AngularJS does translate to a somewhat steep learning curve for the beginner. One of the most difficult AngularJS concepts to grasp is that of scopes - especially when being looked at in all it's different variations.

In an attempt to simplify that I have created an example of a few different uses of various kinds of scopes in AngularJS. Here's a visual depiction of some of the scopes in AngularJS. 

I hope interacting with the above-mentioned Plunkr gives a better idea of how the behavior of the page can vary with the scope used. 

I take no embarrassment in admitting that the term "Dynamic Programming" troubled me for many years. I couldn't figure how these mysterious algorithms worked. It's not that I didn't get the idea of divide and conquer or that of caching. But it seemed too easy that something as that is called "Dynamic Programming" would simply be a combination of "Divide and Conquer" and "Caching prior results of sub-problems". Then I came across this SO post: Dynamic Programming. 

This reaffirmed my suspicion that Dynamic Programming isn't a mystical creature that lives deep in the woods, but instead it is a combination of two techniques. Something that any astute algorithm developer would intuitively come to when writing such an algorithm. Divide and Conquer and then cache the prior results (if that be worth it). Mystery solved. 

Enter Map-Reduce.

If you had a computational problem of scale, and access to multiple processors, what would you do? Wouldn't you divide the problem into sub-parts and branch them off to the processors and recombine the result? 

Isn't this, in a simplistic sense, what Map-Reduce does? Sure, there are all the frameworks built upon this idea that facilitate this, but those are mostly implementation details.

I have been building Django apps using Django templates since some time now. Recently I decide to explore JavaScript frameworks such as Backbone. After reading a number of posts and discussions and experimenting a bit I decided to use Backbone.

The Notes App

At this point I have an example "Notes" app checked into https://github.com/sidazad/django-backbone-stack-1 that shows some of the common patterns that will be required for most web apps. The app has the following parts:

A landing page with a username and password with a login button

A link on the landing page that allows switching between the "Sign Up" and "Login" views

On logging in a User Dashboard becomes visible

The dashboard displays any notes that the user currently has stored

It provides a "New Note" button on pressing which a Modal dialog box shows up

On entering the two text fields on the Dialog Box the inputs are sent to the server to be stored in a database and the newly entered note becomes visible on the web page

  The above interaction will show how to implement the following common patterns with Backbone and Marionette:

  Switching between two views on the same page (Login and Signup)

Navigating to another page in a Single Page App (SPA) while using #s for history

Showing a modal dialog box using Backbone

Automatically displaying an item that has been added to the database

Plumbing

require.js for dependency management

Backbone.js as the JavaScript framework

Marionette: Provides a nice abstraction on top of Backbone

backbone-tastypie to make TastyPie and Backbone play nice

Mustache for client template rendering

Django as the server web framework

MongoDB used via MongoEngine

Code Organization

The Notes app is organized as follows in the top level directory django_backbone_basic:

Client code

static/django_backbone_basic/js contains all Backbone and Marionette related code. 

main.js: The entry point into the JavaScript framework loaded by require.js on the initial request for index.html

app.js: Setup a Marionette application and hook-up with Browser History

router.js: Configure URL routes

controller.js: Triggers events based on the route requested

views/*.js: Marionette Views

models/*.js: Backbone Models

collections/*.js: Backbone Collections

modules/*.js: Modules containing two or more JavaScript modules that are frequently used together for ease of use

Server Code (Django)

views.py: Django Views

mongo_models.py: MongoEngine Models

generic_templates.py: Used to load and return Django templates in raw form (without parsing) as they will be rendered on the client using Mustache

Event Routing and History

A Backbone app needs to react to

URL changes in the browser location bar

Browser back/forward buttons

Signals from a view/module to another

I found the Marionette "vent" object very useful for this purpose. In app.js several triggers were setup as such:

vent.on('show:dashboard',function(x) { Backbone.history.navigate("dashboard"); app.mainContent.show(new DashboardView()); });

The above snippet means that everytime the "show:dashboard" event is triggered #dashboard will show up in the browser location bar and the DashboardView will be loaded into the mainContent region.

Additionally, appRoutes can be set in router.js to invoke the correct function in controller.js based on the change in the browser location bar as entered by the user by entering it directly or refreshing the page or clicking back or forward.

  Problems I Faced

  Template rendering on client or server: It's an open debate whether client or server template rendering is better and it seems the answer may depend on the scenario. By using a plugin like generic_templates.py the Notes app is flexible enough that the templates written in Django can be rendered as server templates (Django) or client templates (with Mustache) hence eliminating the need for duplicating templates.

Duplication of Models on client and server: I have not addressed this issue, hence models/note_model.py as well as mongo_models.py on the server side have a representation of a Note.

Showing a modal region without reinventing the wheel everytime: I came across an elegant solution on a couple of blogs and used the same. Refer to the Using Modals with Backbone and Marionette section in References

Benefits of using a JavaScript framework in addition to Django

Excellent organization or JavaScript code leading to Reusability and Maintainability.

Decoupling from Django's Template Language: More powerful client side template frameworks can now be used leaving Django to function purely as a server framework

Redundant tasks taken care of: Rendering of collection elements in a region in a loop is taken care of when an element is added to or deleted from a collection or it is updated.

Plumbing for SPAs: A considerable amount of plumbing such as browser history navigation is built in.

  Final Word

I hope this post as well as the associated code will be useful to Django developers who want to use JavaScript framework while preserving some of the flexibility of switching between client and server template rendering.

I look forward to comments on this post and to any suggestions for topics to cover in a future post.

  References

  require.js

http://www.youtube.com/watch?v=VGlDR1QiV3A

http://stackoverflow.com/questions/9104899/backbone-requirejs-and-multiple-models

  Backbone

http://backbonetutorials.com/organizing-backbone-using-modules/

http://joshbohde.com/blog/backbonejs-and-django

http://addyosmani.github.io/backbone-fundamentals/#backbone.marionette

  Marionette

https://github.com/marionettejs/backbone.marionette/wiki/Use-cases-for-the-different-views

http://lostechies.com/derickbailey/2013/03/18/event-aggregator-andorvs-mediator-a-tale-of-two-patterns/

http://davidsulc.com/blog/2012/05/13/tutorial-a-full-backbone-marionette-application-part-2/

http://lostechies.com/derickbailey/2012/06/13/why-should-i-use-backbone-marionette-instead-of-%E2%80%A6/

Using Modals with Backbone and Marionette

http://lostechies.com/derickbailey/2012/04/17/managing-a-modal-dialog-with-backbone-and-marionette/

http://jsfiddle.net/niki4810/hD56e/14/

https://github.com/t2k/backbone.marionette-RequireJS

Backbone + Tastypie

http://paltman.com/2012/04/30/integration-backbonejs-tastypie/

https://github.com/PaulUithol/backbone-tastypie

http://django-tastypie-mongoengine.readthedocs.org/en/latest/usage.html#simple-example

Other

http://duganchen.ca/single-page-web-app-architecture-done-right/

http://davidsulc.com/blog/2012/05/13/tutorial-a-full-backbone-marionette-application-part-2/

https://github.com/davidsulc/backbone.marionette-atinux-books/

If you had a choice between using a set of skills you own to create a lucrative business, versus learning a new set of skills in a field you are entering as a beginner but are passionate about, what would you choose?

If you choose to create a business using your existing skill-set, you will learn how to seize market share from your competitors, and in rare cases, create a new market, but you won't grow and learn as much as you would have, had you chosen to learn the new skills. 

What if there was a third choice? To pursue a purpose. One that you truly believe in. Then whether that requires you to use your existing skills or learn new ones.

What would you choose, given that you have to commit to one or the other?

I think the answer comes down to what success means to you. It could mean one or more of the below:

Commendation from your peers

Financial gains

Fame and Power

Whatever it means to your friends, family, peers (in case you haven't figured what it means to you yet)

Knowledge and personal growth

Fulfillment of a purpose - a purpose that defines you - Your Raison d'être

I imagine that the perfect software stack is going to remain a moving target given the speed with which technology changes. But this is where I stand with my current stack of choice. 

In no way am I saying that this is the best web development stack available - that would be a sacrilegious statement given the diversity of good technologies available to us - but this is simply my stack of choice that I find powerful at this time.

  The stack consists of:

Client

require.js

jQuery

Backbone.js

Backbone Marionette

Mustache.js 

Server

Django

TastyPie

PyStache (server template rendering)

Database

MongoDB

MongoEngine

I have created a simple Django App to Sign up users, login and create "Notes" using the above stack. I have tried to cover a number of basic patterns using Backbone and Marionette in the hope that it would provide a solid platform as well as shorten the learning curve for those choosing to use this stack.

I did face a few hiccups and had to collate a number of blogs posts and snippets together to have the above technologies inter-operate, but it was well worth the effort.

The choice of plugins and technologies was made keeping the following attributes in mind:

Ease of switching between client and server template rendering

Flexibility of extending or debugging a plugin

Stability and Community support for plugins

Learning curve