aakashd's blog

To those developers, who are developing or will develop the codebases I will be inheriting in the future.

Inheriting a codebase with bad test-suite is a nightmare. 90% test coverage does not mean anything to me, if I can’t read and maintain the tests.

I want to talk about the perfect test, that I would happily inherit. First, I hope you follow Single Assertion per Test in spirit, if not in letter. Assuming that you do, here is how I would want you to structure my test:

https://gist.github.com/2718348

This is how I use it:

Context - It’s a way to group related tests together. With a context, I can find particular tests easily. Let’s imagine you inherit an e-commerce application, wouldn’t you prefer a context saying “payment settlement”, rather than find the relevant ones through a blob of 100+ tests?

Exact business requirement - A test title, “set status to false”, does not tell me anything I can’t get by reading the assertions. I would appreciate if the test title explains, WHY the code exists, rather than WHAT it is doing. The test body may not be able to explain something like, “a travel itinerary should be archived 3 months after it’s completed.”

<on doing this> - I would like to know the exact steps I need to reproduce the side-effect that we are asserting, nothing more. Having single assertion per test, will make sure that we don’t have bloated setups. This is critical for a maintainable test. 

The emotional energy taken to read & fix a failing test is exponential to the number of lines in that test.

<the assertion>- Apart from “Single Assertion per Test”, following simple readability rules like variable naming, avoiding long statements etc. can make my life so much easier. I would much prefer the second example over the first one:

https://gist.github.com/2718390

or even better

https://gist.github.com/2718428

That’s all I ask of you, focusing on these four aspects of every test. Make sure you write tests like this and I will love you forever.

If you liked this post, please consider…

following @aakashd on twitter

I was introduced to Single Table Inheritance (STI), using Hibernate, about five to six years ago. I don't remember my exact thoughts, but I have pleasant memories of using it. I recently inherited a codebase that misused STI. Irritated, I sat down to write a blog post titled "Why I hate STI." But as I started writing down the reasons for not using STI, I realized that I hadn't re-evaluated the benefits of STI in the last six years.

Let me put myself in the shoes of the Java developer I was, six years ago. The basic assumption behind STI is, multiple classes having a strong is_a relationship will also have very similar fields; therefore the data for all these classes can be stored in a single table. From application modeling perspective, there is absolutely no benefit of using STI. We could easily store the records in different tables as well, but there are considerable benefits of extending the inheritance to database level. For example:

Ease of DB maintenance:

Here, we assume that the schema for all these classes will evolve together. If we have a single table, the db maintenance activities become easy. Like,

adding/removing columns

change in data-types

building/dropping indexes

writing triggers, foreign-key constraints etc.

Database performance:

Assuming we store the records for all types of payments in a single table, a query for "finding all the payments for an order" becomes a single table query. With a single table, the look-ups will be considerably faster due to the reduced number of joins.

Smaller DB interfacing code:

ORM's have certainly reduced the amount of SQL queries we write. Although, I used to write a lot more custom SQL's, than I do nowadays. With a single table containing data for multiple classes, I have to write/maintain a smaller codebase.

These are all the benefits I can think of, but let's see where these aspects stand in the ActiveRecord world. I am taking Rails/ActiveRecord as an example, but the analogy holds true for any modern framework:

Ease of DB maintenance: I have stopped writing triggers, foreign-key constraints etc. With Rails migrations, the maintenance cost has reduced drastically.

Database performance: Still holds good.

Smaller DB interfacing code: I can't remember the last time I wrote an SQL query. On second thought, I did write one about an year ago; but the point is, amount of DB interfacing code we have to write, has almost vanished.

To summarize, there is only one valid reason for introducing STI in your application: DB performance. Although, DB performance is a very important aspect, STI comes at it's own cost. More on the price you pay for this benefit, in my next blog post.

    If you liked this post, please consider...

following @aakashd on twitter

I have worked with legacy Java codebases for nearly five years of my career and to be perfectly honest, I quite happily ignored deprecation warnings for the entire period. However, upgrading multiple Rails applications over the last couple of years, I've learned a healthy respect for deprecation warnings, especially in the Ruby ecosystem.

If there is one thing that Rubyists are not afraid of, it's removing backward compatibility. As developers are not afraid of correcting their mistakes in API design, the quality of ruby library APIs keeps on increasing. But this benefit comes at a hefty cost of code change for every upgrade. For any major version upgrade of a gem, I would end up braking a lot of functionality due to the API changes. If I upgrade my application once in an year, it's likely that I have 4-5 gems due for major version upgrades. The size of your codebase and the test coverage will determine exactly how long an upgrade will take, but it can vary from a handful of days to several months.

Many teams find such rework very expensive and if the test coverage is low, it's a common practice to rewrite the application than upgrade it. Wouldn't it be nice to get the benefits of newer libraries without this effort? There is only one answer, deprecation warnings.

Deprecation warning is a way for API authors to tell us about future API changes. If a method call or class is being deprecated, authors would flag usage of such entity with deprecation warning & alternate API suggestions. For every frequent minor upgrade, if you change the API usage as suggested, you will have a smooth major version upgrade. As the cost of development and QA is spread over a complete year, you would hardly notice this cost saving the costly rewrites.

Certain guidelines to follow here would be:

upgrade frequently; check every fortnight if any of the gems in your application can be upgraded

have your build fail on deprecation warnings

never commit code with deprecation warnings

The key discipline to follow here is, "upgrade frequently and never commit with deprecation warnings". You will be surprised, how easy it becomes to keep your applications up to date.

  If you liked this post, please consider...

following @aakashd on twitter

Recently, after inheriting a codebase that was difficult to read and maintain, I thought of creating a simple list of tips & tricks for developers. The list is very primitive, but just following these practices can guarantee a decent quality codebase.

If you can think of anything that I have missed, I would be happy to add that to the list.

Writing Tests

single assertion per test (ref: http://blog.jayfields.com/search?q=single+assertion)

tests should  be in their proper context (ref: https://github.com/thoughtbot/shoulda-context)

test name should describe the behavior being asserted

build should not give any warning

use libraries (like shoulda) to reduce the verbosity, e.g. should_not allow('invalid').for(:payment_type) rather than writing a complete test

Writing Code

method should not go beyond 7 lines (ref: http://c2.com/cgi/wiki?FewShortMethodsPerClass)

avoid static methods (User.find_by_name is a class method, User.format_date is a static method) (ref: http://blog.jayfields.com/2007/04/ruby-class-methods.html)

do not use magic strings/numbers. (ref: http://c2.com/cgi/wiki?MagicNumber)

do not pass around hashes. Identify the behavior associated with that hash and build appropriate objects.

Process

commit and push every 30 minutes. frequent commits => Less merge conflicts => less rework

never push failing build

never do partial commit. Partial commit is committing only part of the changes on your machine. The build may pass on your machine but not on others.

Some Reading Material

http://c2.com/cgi/wiki?SingleResponsibilityPrinciple

http://c2.com/cgi/wiki?LawOfDemeter

http://c2.com/cgi/wiki?SingleChoicePrinciple

Most Important Practice: Once in a while read your code aloud. If what you are reading makes a lot of sense to you, the developer who is going to read it 6 months down the line might be able to understand it.

  If you liked this post, please consider...

following @aakashd on twitter

I am not trying to say that money earning is unethical or immoral. But before I put forward my opinion let me cite few examples I have seen around me.

A freelancer works for 8 hours and creates a mediocre application for Rs.10,000/- (billing $27.77/hour) for a company. The company sells that piece of crap to a Bank for Rs.2,25,000/- (billing $625/hour).

A multinational bank needs simple log parser and web based reporting application working on top of the summary of data derived out of that parsing. Something pretty much possible with simple PHP script with normal desktop as well (and the developer knows it). This consultancy asks for a Dual Quad Core machine, 16 GB ram and develops code in Java so that it looks complex and charges Rs.2,50,000/- for it. The max effort with all beautification and setup is not more than a week. Billing rate $125/hour and am not even considering the infrastructure cost bank will incur.

A freelancer is given a job to be completed in 6 weeks, but its so simple that he completes it in 3 weeks. The company who hired the freelancer, warns him not to do such thing again, as it exposes their inflated estimates to client.

This one is worse - A business analyst tells developer that the application user interface should not look simple and easy to use. It should be very complicated and non intuitive, so as to look difficult. This would make them bid more cost as the product itself looks complicated and difficult to develop.

This is the worst - A company always leaves few known bugs in its products, so that they can charge for maintenance. Also the code is kept cryptic and undocumented (with the kind of developers they have, may not take any effort!) just to make it difficult for any third party to understand it.

Though I cannot tell the real names of developers and companies involved in these frauds (for obvious reasons), you probably know that these things happen. I have also come across colleagues who told me, that I am a moron and should never get into business, for this is the business. Apart from a gut feeling that these things are wrong, I have some serious concerns over how this affects IT Industry and the society as whole. Lets take a scenario and the ill effects it has.

This insane increase in the cost of software development increases the operational cost of our clients. So it shouldn't surprise us when we have to pay more for Hospital bills, pharmaceuticals, transportation, loan interest rates, rent etc. Also when the FD interest rate drops or the Mutual Fund NAV drops; we should applaud our contribution as well.  This list can go on, but crux is, that the inflation we cause will trickle down to us directly or indirectly.

Creating complicated software to charge more, is a subset of previous point but it has another aspect as well. The very essence of automation is to get rid of mundane tasks. If the applications we develop just creates another complexity, it again increases the cost of operations and future enhancements to the application. Such architectures will create information silos, creating big hurdles in the growth path of organization. I have seen a single company storing and maintaining millions of records of same data (pertaining to its business) at more than 5 places, cause the applications were never developed to talk to each other.

Might sound idealistic, but as an IT professional I see myself as a proponent of automation and seamless integration. But these practices would create mental and monitory apprehensions for the acceptance of technology and change in traditional industry. I think its time to change this, either by preaching or by direct action.

  If you liked this post, please consider...

following @aakashd on twitter

At any web site we have images with different sizes for different placements. For example size

50 x 50 for thumbnails

150 X 150 for medium sized images to be displayed in search results etc

400 x 400 for large sized images to be shown in slide show

One way is to process images immediately on upload, where we create all the required sized images synchronously with fixed path/filename. There are loads of ways to do it, so I wont go into it. But let me describe the problems that tag along this approach.

Assume a scenario, where we had decided 50x50 dimension for thumbnails and now we have 2 million thumbnails created over the period of 2 years. Good thing, and good time to revamp our design. As per new design, the images would need to be of size 75x75. OMG! What do we do now. Simple solution is to run a job to change all existing images, DB paths, code base to work with this new dimensions. Those who can imagine the scale would tell you, that its a nightmare.

Do I have a universal solution? NO. But I have implemented a simple, nearly as efficient and much more maintainable approach using Rails and Fleximage. How to use fleximage? go through the wiki. Here is the basic approach...

Let fleximage store the image as it wants (or how you want), the default path is the [./public/images/uploaded/yyyy/mm/dd] and the file name is image id in DB. But you can change these as you want.

In the Image controller when you receive request like /photos/1.png, you render the original image.

When you receive request like /photos/1_50_50.jpg, use the 50 x 50 to create the thumbnail. Sample code given below...

# GET /photos/1 def show idsplit = params[:id].split('_') image_id = idsplit[0] @photo = Photo.find(image_id) if idsplit.length > 1 idsplit.delete_at(0) render :inline => "@photo.operate { |p| p.resize '#{idsplit[0]}x#{idsplit[1]}' }", :type => :flexi end end

Believing that you have enabled caching for show action on PhotosController, the rendered image will be stored in your public folder in whatever cache location you have specified.

The next request to photos/1_50_50.jpg, would directly served out of your cache with performance hassles :)

Thats pretty much how I have done it. The ruby/rails pro guys out there, please pardon the code its taken from one of my spikes with Fleximage. About who makes the decision of, thumbnail for a product should be of 50x50, I have created helpers which give me path to thumbnail images on passing photo object. You can pretty much encapsulate any way you want.

I have always had this complaint against the web designers around me. They would use 28 inch iMac and Adobe Photoshop to create the mocks; which are most often seen on 14 inch CRT in IE 6. Everything looks goody goody to them; but when it goes through our IDEs, few of the effects are not as per their expectations. The very definition of what's above fold and below fold changes because of the screen size. Recently I got answer to my second issue of screen size - browser size, another brilliant tool by Google. On providing your web page URL, it would show you a graph as to which part of your page is visible, to how much percentage of users (the dataset is Google visitors). This is useful for ensuring that important parts of a page's user interface are visible to a wide audience. I won't talk about it more, just check the about page. But take it with a pinch of salt, as for India centric site like ours the ratio of users with smaller monitors would increase a lot. About the issue of using Photoshop for designing web designs, I found a nice article by Meagan Fisher. It talks about why making your mocks in browser is the way to go. I would just summarise the benefits...

Clients can see how it looks in different browsers.

Client can check interactions like on hover, visited links behaviour etc.

In case of missing images or disabled JS, what's the impact on pages look and feel.

How the page would look while loading when part of the content is loaded (specifically image and flash components)

Designer gets to check the feasibility of his ideas

CSS is almost ready when mocks are complete

I have always seen HTML attribute class used for tagging elements with css class name. To an extend that, I always believed the class attribute was meant to be used for CSS. Over the period, I started noticing something fishy about this class business. First was when I used the method document.getElementsByClassName(). If I am using class attribute for css, then why are we polluting the JS API with something which was meant to be for styling only. My styling changes and then my JS code is broken. That was my first interaction with the term selectors, and I got to know how CSS works. The classes we specify are just used as selectors by the browser, to give particular styling to the elements. So all this time I was using CSS, all I was supposed to be doing, I was just tagging elements. This brings me to next question. If I have a red header in left column, should I tag it as class="header left" or class="header red"? Both approaches have their pros and cons. Let me take the first one, which I will call purist.

Purist : The elements should be tagged based on their role in page and not by their styling requirements.

In case of three column layout, even if both left and right side column headers have same styling, the class names would still be class="left_column header" and class="right_column header" rather than class="header green". This sort of approach would increase the number of classes one needs to define in css file. Every element will be tagged to describe what that element stands for, and css along with other frameworks, uses these tags to identify the element to which style is to be applied. As said before, it will certainly increase the size of the css file, but at the same time makes maintenance very easy. In a scenario, where style of an element is to be changed, only the css file needs to be updated. If we had gone for the "green header" approach, we will need to access every page where this class is mentioned and change it to "red header".  Also where templating needs to be done, such approach is the only option.

Popular : Element is tagged based on the styling requirements and not on their position/role in the page.

URL rewriting is a very regular way of making your URLs user/seo friendly. But so is url based authentication. URL based authentication, allows us to specify url pattern and roles required for these patterns. Both these functionality are request filters, and obviously cannot be executed at the same time. Thus next question, which should be executed first. I saw ppl doing it either way, but without much justification. After thinking a while I decided to use UrlRewriter first and then the auth filter for following two reasons.

As my application evolves and I get richer data, my urls would change to encompass more data. Say I get more detailed geographical data, and I want to add that to my URL (obviously). I auth was to get executed before rewriter I will have to change both auth and rewriter. Thus keeping auth after rewriter would make my authentication impurvious to any changes to user facing Url structures.

On similar grounds, keeping UrlRewriter after auth would force us to mention both the original as well as rewritten URLs in auth configuration. Keeping only user facing urls in auth configuration is just little less than not keeping th authentication module at all.