Floating Point

It is the ability to substitute one type in place of another.

Why Polymorphism?

This allows us to write code in a generic way. We can write our code against an expected contract without worrying about all the details of an implementation. For e.g. you can write some code against a generic Vehicle type, and others can use it with more specific types (e.g. Car)

It provides more flexibility and reduces code repetition to support similar behavior of different objects.

It allows us to modularize our code better. We can program against a type without having to have a source code dependency on concrete types. This in turn allows the concrete types to be developed independently.

How is Polymorphism achieved?

Using interfaces in languages that support it - where you can program against an interface and at runtime other concrete types can be used in it's place.

This can be used using inheritance where you can program against a parent type, and at runtime you can pass in other concrete sub types.

Duck typing in languages with a dynamic type system. You can pass any object that contains the expected methods at runtime and the program will use that. This is similar to using interfaces for Polymorphism, except the language does it automatically.

Method overloading is another form of Polymorphism. You can offer the same interface to the consumer of a class, but support different types of input objects.

Java generics offer a form of Polymorphism where data structures and algorithms can be written in a generic way to support various types

What is static vs. dynamic Polymorphism?

Dynamic Polymorphism occurs at runtime where the concrete behavior to use is determined at runtime. An example of this is: invoking a method on an interface in Java, at compile time, the compiler just makes sure the types are compatible. But linking the behavior to execute happens at runtime; the runtime looks at the concrete type it receives and links the interface method call to the corresponding method call in the concrete type

This is an old unpublished post. Published now.

Good design can make a product look and feel amazing. As a developer who likes to build web apps, this is a skill I value and want to get better at.

I do a lot of information hunting and reading to get better at design. I am just going to curate that information in this post.

The first most amazing "Design for Developers" video I saw: Design for Developers: Making your Frontend Suck Less

Today I found this and this seems similar plus a few new things: Design for Developers

This is an old unpublished post. Published now.

Bounds are important. Unbounded or only partially bounded resources and operations pose a risk to latency and availability of your service.

* Bound resource usage (e.g. limit thread pool size) * Bound queries (e.g. put a where condition limiting the date range, don’t make it open on one side or worse both sides; if it makes sense use the"limit" features in queries) * Bound time spent waiting on a dependency

This is an old unpublished post. Published now.

Writing good code:

Clean code: The content is excellent. Each section is short and to the point. This is a very pleasant read, and the content applies to most if not all languages. 

Mastering your tools: Java

Effective Java: The title is an excellent description for what this covers. It helps you understand how to use different Java constructs effectively, common mistakes to watch out for and how to avoid them, etc.

Java Concurrency in Practices: Understand how to write concurrent programs, concerns that you need to think about, and how to use Java correctly to write safe and performant concurrent programs.

Software Engineering:

Release It: Excellent book on how to build non-trivial applications (applications that are not just 1 program running on 1 computer). It covers the entire life cycle of planning, developing and supporting software.

Site Reliability Engineering: This book will teach you how to manage cloud or enterprise applications effectively. It’ll help you understand how to ensure the application is healthy and keeps going.

Refactoring: This is a very tactical book. It teaches you methods to transform code - how do you work your way out of common anti-patterns in code. I love the small chapters that are example driven.

Test Driven Development: by Example: This is a very practical and approachable book. It contains two simple and clean examples that show you how to start using TDD in your practice. It’s a very easy read.

Working Effectively with Legacy Code: Practical book that describes various problems you face in legacy code and explains ways to work yourself out of those problems.

Interview Preparation:

This is an old unpublished post. Published now.

Every technology out there is trying to simplify something. It is providing you an abstraction that offers you a simple interface to achieve complex results. Understand what the technology does. Understand why it works. This will allow you to simplify your designs. After all “Data Structures, not Algorithms, are central to programming ~ Rob Pike”.

Kafka and Kinesis provide you a log-like storage system. It’s easy to keep adding things to the end of a list. You can do this at a high speed. However, this structure makes it hard to perform any smart queries on the data.

The importance of code review: skipping to do a code review because it looks involved, could end up costing you — since you have to add to that code and it wasn’t well structured to begin with.

One good heuristic on when something needs to be restructured is: if there is too much going in a class to keep in your head, call that out in your review — ask people to break it out into a separate class.

This is also the “S” in SOLID — Single Responsibility Principle. You can get technical and argue what a responsibility is. But if you can’t keep everything going on in the class in your head, that’s too much. If the tests for that class look large and/or complex, that’s too much.

The smaller class will ensure the code is more comprehensively tested — since you can see all the different behaviors. Each class will be a lot easier to understand too.

How do you ensure that you don’t have class explosion? Class explosion is bad when you have 1 object that needs to depend on 20 objects to do its work. If 1 object, only depends on 5 — it’s small enough to wrap your mind over how it works. Each of those objects can depend on 4 other objects and in the context of each class it’s easy to understand what they do. This is good.

I’ve been thinking of code reviews lately and how can I be more effective when doing them. Here are my thoughts.

A code review has 2 main goals:

To improve the quality of the code being reviewed.

To spread knowledge.

How do you improve the quality of code?

You improve the quality of code by taking it closer to ideal code. So what is ideal code? Ideally designed code:

Works well today

Is easy to change for tomorrow

Does not create an operational burden

How do you spread knowledge in code reviews?

Just the fact that someone else reviews your code means they have better knowledge on how a feature works. You are improving the bus-factor of your software/product.

Code reviews can be an educational experience for new developers who might not know the best way of doing things. The change you are suggesting might be small in the scope of the review at hand. But that small change, as it carries through their career can have a positive impact on many other projects to follow. When you provide educational tidbits, make sure to specify the reason why someway is better — that is what makes it educational.

You can pick up on nice ways of doing things from seeing how others have done something.

In essence:

Give feedback, that will:

Make software work better today. For example: identifying bugs, missed edge cases, performance problems etc.

Make software easier to change for tomorrow. For examples: identifying architecture problems, promoting good code structure, following code conventions etc.

Decrease operational burden created by the software. For examples: Recommending good logging behavior, suggesting alternatives to things that would require manual maintenance etc.

I was recently speaking to someone who wanted to setup testing in a new organization. Here is what I said. I think it captured one essence of testing well, so I wanted to write it down - so I can remember and others will benefit.

There are tests at multiple levels for software: unit tests, integration/functional tests, end to end tests

Each layer provides additional protection. They have different trade offs. Unit tests are very quick to run. But they won’t catch bugs outside of a unit. End to end tests are very close to realistic use cases. But are slow and generally fragile. So they are expensive to maintain

Just came across these 2 tools today. They'd be a great addition to any Java project.

Mutation testing with PIT

In Part 1 of this series we learned how to suspend execution of a function and then continue later. In Part 2, we learned how to provide the function a value mid-execution.

With these 2 things, we can control the flow of code within a [generator] function from outside of it. We can tell it when to run, when to pause and when to continue and with what value.

Putting that in the context of async code. You could have some async operation in your [generator] function. You tell it when to run, when to pause (when it hits the async code), when to continue (when the async operation is done) and with what value (the results of the async operation).

How would that look in terms of code. Let's look at an example:

function* run() { console.log('started; will wait for 5s'); yield delay(5000); console.log('running after 5s'); }

The above code can be asynchronous. You execute the first console.log. Your function gets suspended for the delay operation. When the delay operation is done, it continues. We want to specify how long the delay has to be, so we provide it a value ("5000" above).

Traditionally the above code would have been written like this:

function run() { console.log('started; will wait for 2s'); delay(5000, function () { console.log('running after 5s'); }); }

Isn't the transformation neat? There was no callback. The two things would do the same thing. Neither of them actually block the current thread. It is just one of them looks synchronous and doesn't use a callback making the code easier to read and the other one uses it.

Well, I lied (a little). As nice as the first code snippet is, you need to add some code to make it work. What we have created is a generator function. A generator function by itself doesn't do anything. It just returns an iterator.

The iterator has to be managed by someone. That someone will determine when to run, when to pause and with what value to continue running. We need to write that code.

var gen = run(); var asyncFn = gen.next().value; asyncFn(function (err, res) { if (!err) { gen.next(res); } });

We call the generator function and get an iterator back. We begin execution of the function and proceed till the first yield. What we yield is an asynchronous operation. We want to continue when the async operation is done. In node, we do that using callbacks. So the asyncOperation that was yielded needs to take a "done" callback, so we can just hook to it and know when we should continue the iterator.

So what we want is "delay(5000)" to return an async operation that takes a "done" callback which will be called on completion.

Let's write out the delay method that would do that.

function delay(n) { return function (done) { setTimeout(done, n); }; }

So when delay(n) is called, we return a function that takes a done callback. The returned function's body does all the work and knows to call done to signal completion.

This way the code managing the iterator knows when to continue execution of the code in the generator function. This is how we make the async code look synchronous. The obvious benefit is you can avoid the callback hell scenario which node is infamous for. The other benefit is you can use try/catch to catch errors in the async operation. The iterator has an error method "iter.throw(errObj)" calling which throws an exception from the yielded point in the generator function. You can wrap that in a try/catch and handle the exception.

This is a continuation of Part 1. Part 1 talked about how yield suspends execution of a function. If you don't know what that means, you should read Part 1 first.

You can not only pause function execution mid point with yield, but you can also trigger when to resume and with what value.

Let's look at an example to make this clear:

function* square() { var s = yield null; return s*s; } var iter = square(); iter.next(); // Runs the code up the yield. var result = iter.next(5); // Resumes from the yield statement, however the yield also returns // the value 5 within the function, since it's being sent in from here // This runs up to the return statement console.log(result.value); // Returned value is 25

Now, lets try an exercise. What happens here?

So, what is happening here? Calling the generator function gives the iterator (iter). The first iter.next() runs the code up to the yield statement. Note that the assignment comes after the yield is evaluated. So the assignment isn't evaluated in the first iter.next() call.

The yield is expected to return a value - since that value is assigned to the variable s. How can we make the yield return a value? By calling iter.next with the value. This returns a value from the place of the last yield and continues execution of the function till a yield or return statement. This is what we do above with the iter.next(5) call. We send the value 5 back to the function as the result of the previous yield and continue on.

Here is a small test to verify your understanding. What is the output of this?

function* wrapFunction() { var fn = yield function () { console.log('I was returned'); }; fn(); } var iter = wrapFunction(); var origFn = iter.next().value; var result = iter.next(function () { console.log('Wrapped the original function'); origFn(); }); console.log(result.done);

The expected output:

Wrapped the original function I was returned true

The 'yield' feature is being introduced in ES6. This is great news as it enables writing async code that looks synchronous and you can even use try/catch to capture errors in your async calls. But before we get to how, let's look at how 'yield' works.

Where does it work?

The yield keyword works only within generator functions. Generator functions were introduced in ES6. I know it sounds fancy, but it's actually simple - this is a generator function:

function* doWork() { var a = 1; var b = 2; yield 'set values'; yield 'returning values'; return a + b; }

It looks just like a normal function. There are two differences:

You declare it with function* instead of function

You can have the yield keyword within it

Okay, so how is it different?

When a function hits a return statement, it exits the function. It is done! With generators, a yield statement is similar. You return a value from the yield statement, except you are not done. You just say "I am done for now, let's continue later". Do you see the difference? You have suspended execution. The next time, you can just continue from where you left off. This was previously not possible in JavaScript.

How does that work?

The following code snippet will make it clear.

var worker = doWork(); do { var work = worker.next(); console.log('worker:', work.value); } while (!work.done)

Calling the generator function ("doWork" above) returns an iterator. Don't let that scare you, all it means is it gives you an object that has a next() method. You can call next() to get the next value out of the generator function.

So in the above context, worker is the iterator returned. You can call next() on it to find the value returned before execution of the generator function was suspended. Every time you call next(), the function runs up to the the next yield or return statement in the generator function.

The next() function returns an object that has 2 properties: value and done. "value" Represents the return value before suspending execution. "done" Represents that the function has actually returned (and not just yielded), it is done! it cannot continue execution. How does value get it's value? It is the value that is yielded or returned. Similar to "return <value>", you can "yield <value>". That returns the "<value>" and suspends the function's execution.

Can you guess the output of the above code?

I'll wait.

...

Alright, so it is:

worker: set values worker: returning values worker: 3

The first time you called worker.next(), it ran up to the first yield statement. It returned the value that was yielded: "set values".

The second time you called worker.next(), it ran up to the next yield statement and returned the value "returning values".

The third time you called worker.next(), it ran up to the return statement and returned the sum of "a" and "b". Since this is return statement, the object returned by worker.next() ("work") has the done property set to true. This causes the loop to exit.

And now that is how yield works.

Suspending execution is just one part of how yield works. There is another aspect to it where your functions can take inputs in the middle of the function, so not just in your function parameters. But also from some mid point in its body. That's where stuff gets really interesting. I'll reserve that for part 2 of this post.

Once we understand part 2, it will become easy to see how this can be used to make async programming in JavaScript nice and fun. And that will be part 3.

Alright, I am going to sign off for now. Why don't you check your understanding in the meanwhile by predicting the output of this:

function* doWork() { var sum = 0; for (var i = 0; i < 5; i++) { sum += i; yield sum; } } var worker = doWork(); var work = null; do { work = worker.next(); console.log(work.value); } while (!work.done);

Setting up nginx:

https://www.digitalocean.com/community/articles/how-to-install-nginx-on-ubuntu-12-04-lts-precise-pangolin

Simple instructions for setting up nginx.

Setting up node.js (using nvm; dev + prod instructions):

https://www.digitalocean.com/community/articles/how-to-install-node-js-with-nvm-node-version-manager-on-a-vps#installation

Installing node.js this way has the advantage of supporting multiple versions (including future versions) and switching between them easily. You can also use this to run specific node applications using specific versions.

Hosting multiple node.js apps on a single VPS:

https://www.digitalocean.com/community/articles/how-to-host-multiple-node-js-applications-on-a-single-vps-with-nginx-forever-and-crontab

Instructions on hosting multiple node.js applications with nginx as the reverse proxy. Instructions also cover using "forever" to respawn node processes if they crash and setting up a cron task to start up the node applications on reboot.

Note:

I have to say, the articles and documentation at ditialocean are very good. If you are wondering how to setup something in a cloud service, do check them out: https://www.digitalocean.com/community/

Writing Modular web applications with Node.js and Express

I watched this video long time ago and I guess the ideas here just passed me by. I was talking to a friend recently and he told me how their node.js development has improved since following the advice from this video. I decided to re-watch the video.

This is definitely good advice on creating maintainable express.js applications. The key take aways for me from this video were:

You can create sub-applications and pass these as middleware to your main application using "app.use(subapp)". The subapp can contain its own routes etc. The different zones of your application can be cleanly separated. This is mentioned around the 5:00 minute mark in the video.

OAuth2 is a draft standard on how to perform delegated authentication/authorization on the internet. It is already used by many popular sites like Google, Facebook, Twitter etc.

I have had a hard time understanding how OAuth2 works and why we do certain things in the process. This is a note to myself and also for anyone else who is struggling to understand how and why it works. Let's get started.

Introduction

OAuth2 allows you to:

Authenticate users: The nice thing about this is you don't have to implement a user account management system, you get it for free from a 3rd party service (like Google).

Access user data from a 3rd party service: With OAuth2, users can authorize your application to access their data in another service. Your application can read/edit a user's data in their Google, Facebook, Twitter etc. accounts.

In both cases there are 3 parties involved:

The User

Your application ("the OAuth consumer")

The 3rd party service ("the OAuth Provider") - which will authenticate the user + where you will access user data from

This post focuses on the "Access user data from a 3rd party service" part of OAuth2. The "Authenticate User" is a subset of this. Once you read the post, you will be able to figure out that part yourself (may be with some head banging - I just want to keep the post focussed). All the examples below use Google as the 3rd party service

The high level idea

User visits your site and wants to log in. He clicks the "Sign in with Google" button. You have specified all the resources you need permission to from his Google account.

The user is redirected to Google, where he can log into his account. Google will prompt the user if he wants to allow your application to access the requested data. After he says yes, Google will redirect the user to a page on your site along with a token that represents who he is. If he says no, Google will redirect him to a page on your application and pass an error message to you.

You can use this token to access the user's data from Google using the Google APIs.

Since access tokens represent access to user data, they are short lived for security reasons; they expire after a short period. Your application would have to get a new access token after the last one has expired to continue accessing the user data. However, once the user grants your application permissions, they will not be prompted to authorize your application again - unless the permissions were revoked.

The infamous flows

The way the three parties (user, OAuth consumer and the OAuth provider) communicate with each other changes based on the type of application. These are referred to as different flows. I'll just focus on the two most common flows for web applications.

Client Side Flow

You would use this flow if you only need to make API calls when the user is actually on your site, i.e. if the user isn't actively using your site, you don't need to access to their data from the 3rd party service.

Here is what happens in the client side flow. I'll use Google as the 3rd party service:

User clicks "Sign in with Google". [User]

User is redirected to Google, where they authorize your application. [3rd Party Service/User]

Google redirects the user to your site with an access token in the URL hash. [3rd Party Service]

Your JavaScript code would read the access token from the hash and verify it by sending a request to Google. [Your Application]

Once the token is verified, you can make API calls sending the access token to Google to read the user data. [Your Application]

Server Side Flow

You would use this flow if you need access to the user's data even when they are not actively using your site. An example may be, your application aggregates particular posts from the user's Twitter stream. Even when the user isn't actively using your site, your application needs to do work using their data.

You may use the client side flow to do server side communication as well. Once you have the access token, you may use that token for as long as it is valid. So once you get the access token from the client side flow, you can use it on the server side.

However, there is a more secure and better suited way to use OAuth on the server side. Here is what happens in the server side flow (aka one-time-code flow):

User clicks "Sign in with Google". [User]

User is redirected to Google, where they authorize your application. [3rd Party Service/User]

Google redirects the user to your site with an access code in the URL hash. [3rd Party Service]

Your JavaScript code sends the access code to your server. [Your Application]

Your server sends the access code to Google and gets an access token and a refresh token. This access code can be exchanged for the tokens only once. So even if this access code was stolen, as long as it had been used once - it is worthless to the stealer. [Your application/3rd Party Service]

Your server verifies with Google that this is a valid access token for the user and your application, and then starts using it. You may wonder why do you need to verify this is a valid access token, since you just got it. Remember this access token was exchanged for an access code. If an access code for a different application was used, you would get a valid access token - but one that does not belong to you. This is why it is important to verify the access tokens in this case. [Your application/3rd Party Service]

As you remember, the access tokens are short lived. So once the access token expires, you can go to Google with the refresh token and exchange that for a new access token. You can keep doing this till the permissions for your application are revoked. [Your application/3rd Party Service]

OAuth2 and Security

When an access token is stolen, another application can act as your application and access the user's resources in the 3rd party service. So it is important to ensure access tokens are handled in a secure manner and that they cannot be stolen. Here are some some security measures to protect access tokens:

Use HTTPS in all transmissions of the access token

When using HTTPS, make sure you validate the SSL certificate of the destination. If the certificate is invalid, do not proceed.

Do not store tokens in plain text cookies

Do not pass tokens in page URLS

Make the tokens short lived. This would be implemented by the 3rd party service issuing the token.

References

The information I have described above comes mainly from these pages. They are more in-depth and a good read once you get the high level picture from this post. You can see the actual request/response/code in the pages below. Check them out.

OAuth in general

http://aaronparecki.com/articles/2012/07/29/1/oauth2-simplified

http://www.slideshare.net/aaronpk/an-introduction-to-oauth-2

Generic Sign in with Google (using Google as an authentication mechanism for your app)

https://developers.google.com/accounts/docs/OAuth2UserAgent

https://developers.google.com/+/web/signin/server-side-flow

https://developers.google.com/+/web/signin/client-to-server-flow

Sign in with Google (the button)

I have found an amazing way to write node programs. This may be common sense to some developers. But I was so glad when I realized this.

The Past

I used to start off with a web application in mind when I tried to solve problems. You need a wordpress auto-publishing system? Let me build a web app for that. My thought process would go something like this "If I have to make this a web application, I first need to pick a good web framework. May be I should just use express..no, too barebone. I need to go faster, but something lightweight - locomotivejs? The model layer is left to me, I'll use mongoose. After tinkering around for a bit...I don't like how much boilerplate code I am writing..I should try compoundjs..scaffolding is cool. Documentation seems full featured. Oh man, I have to learn yet another framework? I don't like this anymore...". You get the idea. I would work on the plumbing for great lengths. I would overrun my initial estimate and I would have to pull the plug on that hobbyist project at least temporarily. I would have put in way more effort than necessary and I wasn't solving the main problem.

Now that was just horrible. I would feel bad after every abandoned project. I even considered abandoning node.js and learning Dart (gasp!). The picture wasn't looking very pretty.

The Turning Point

I got an email from someone at work with a bunch of images. Now all the images had a prefix "16x16". I did not want those prefixes. Normally I would just go about renaming files manually. Or if there were a lot of files, I would google around looking for a utility or learning how to do that in the command prompt.

Instead this time, I looked at it and thought "hmm, I could do this from the node REPL". I opened up the command prompt, typed "node" and then this:

fs.readdirSync(".").forEach(function (file) { fs.renameSync(file, file.replace("16x16", ""); })

That was it, all the files were renamed. Boy, did I feel like a rock star for something so small.

The Realization

I realized this was the key. I should be writing single purpose scripts when I start off a project. Writing scripts in node.js feels amazing. They are written in lovable JavaScript and you get an amazing API to interact with the OS and the network.

So you have a nice language, an amazing and powerful API. Now all you need to do is write a small program that targets your problem. If things do grow out and you need to make a web service out of it, you can just "require" the small modules you wrote within your web framework making the changes you have to.

As you write more node code, you realize JavaScript is a language where it is super easy to get yourself choked. You need to keep things small and clean to write software that you plan to change more than once. Writing small, nicely encapsulated modules and getting them to play well with each other is the key to writing maintainable node apps.

tl;dr;

When you are building out a node application or solving a problem, start with a small, single purpose script. You work on the core problem from the beginning. There is a greater chance you will actually solve the main problem and when you do, you would have done it sooner. This alone, makes you feel great.