Karan Kurani

Note - This guideline is most useful for the bottom 99% of the practising software engineers. So if you are part of the remaining 1% - you can safely skip the this post.

There is a surprisingly easy hack that you can apply to increase your value generally in life. Pick up more skills. It sounds obvious when put on paper but there is one subtlety which makes it a “hack” in my opinion.

For this point we turn to the excellent Scott Adams’ book - “How to fail at almost everything and still win big”. Speaking about being successful, he says -  

“… I tell them there’s a formula for it. You can manipulate your odds of success by how you choose to fill out the variables in the formula. The formula, roughly speaking, is that every skill you acquire doubles your odds of success.”

He goes on to mention that the level of proficiency for a skill is not mentioned because -

“… you can raise your market value by being merely good - not extraordinary - at more than one skill.”

“To put the success formula into its simplest form: Good + Good > Excellent”

This subtlety makes it very easy to execute. You don’t need to be extraordinary, you just have to be ordinary/average. Hence, if you are an average software engineer and you have any of the skills mentioned below -

Good at drawing 

Public speaking

Managing people 

Have a knack to pick up people’s emotions

An eye for design in product

Spot problems in operational processes

Shoot, edit and/or make videos

Make original music

Sing 

Writing essays/blog posts

Or anything else

You are immediately more valuable to yourself and your organisation vs someone is who is a very good software engineer only. So if you are a software engineer who can think of product ideas and execute independently (remember you only have to be of an good/average skill in it) - you have more than doubled your value.

Leverage that skill.

This is a short excerpt from a book I am writing - self.debug. Its about using emotions and mindfulness to become a better software engineer. 

It has a free option if you want to use one - but I absolutely love it when readers pay for it - nothing encourages an author more than people willing to pay for their work. Please download it and help make it better by providing feedback.

And why you should be careful about anything that ends with -er. Before I go any further, please go and read Steve Yegge’s blog post on the overuse of Nouns in Javaland.

Most coders I know do not give naming things the importance it deserves. It is highly critical to get the class/method name right the first time. The name will affect the way you structure your code base, think about it further down the line, how you refactor and extend it. Nothing creates technical debt faster than choosing a wrong name for your class. It will come back and bite you very painfully.

The name of a class will determine how you delegate responsibilities to that class. Let me illustrate. Lets say you are designing the iOS client for Shoutt. The client needs to establish a connection to the server, fork over the data and then handle the response. At this point many coders will say - “Aha! What we need is a ConnectionManager which manages all these tasks.” So we go ahead and create that class with the required functionality.

It works fine until we realize that the app requires multiple simultaneous connections to the server. Now because you have a ConnectionManager, you go ahead and introduce an array inside the class which maintains the state of all the open connections to the server. The code is a tad little bit more complex because each connection is established by a different object in the client and the response needs to be routed to the appropriate caller.

Fine so far… Oh wait, but now the client also needs to handle retries in case of failure. What do you do if one of the connections fail? You introduce a hashmap in ConnectionManager which maintains the number of retries a particular connection has made to the server.

Oh crap, what if the object establishing the connection wants to end it before MAX_RETRIES is reached? So now you have a connection_id which is stored on each view and is used every time you want to cancel the connection. Also, did I mention that sometimes connections need to be asynchronous and sometimes synchronous (this requirement crops up two weeks into building the app btw). So now, in your code base, when you want asynchronous connections, there is an event thrown in the ConnectionManager with the connection_id that shows the response. But in case of synchronous connections it returns the response immediately. So you have two methods on ConnectionManager object called establish_sync_conn() and establish_async_conn().

At this point, your ConnectionManager object has an array, a hashmap, probably another hashmap to track connection types and two different methods just to create a connection. Its a very complex thing to look at. Even worse is the fact that if someone else looks at your code, it will take them a while to figure out whats happening where and how everything is linked together. It also makes you prone to make mistakes and have more bugs/complexity in your code.

What would have happened if you had named the class just Connection instead of ConnectionManager? Just because of the name being Connection you think of it as a proper object (like a proper connection - think of it as an information pipeline between the client and server). You need multiple connections? Just create another object of connection. What about a single connection for the entire app? You just create a global connection variable which has that object. Retrying is also maintained by each connection object independently. How about max retries? You just have an internal max_retries variable in the class. So each connection maintains it’s state internally. Since we have separate objects for each connection, each object can just handle that connection on its own and terminate it without worrying about whether there are any other connections or not. What about the sync and async type of connections? The answer is in the word type. You pass in the type of connection you want in the constructor and the object is setup accordingly.

Obviously the ConnectionManager could be designed differently and made cleaner, but giving the name Manager makes defining the responsibility of the class ambiguous. On the other hand, naming the class Connection would make its responsibility clear and unambiguous.

Anything that has the name manager (or anything else that ends in -er) in it is usually (not always) a code smell that you haven’t designed your application properly. Eliminating that type of class will automagically lead you to write much better code and keep things simple as you extend your application.

Picking the right name helps you keep your code structure clean. It also make it easy to extend as well as refactor. So next time you are naming a class, please give it an extra 5 minutes of thought before you start coding.

========

[Note - This was originally posted on March 30, 2013 on my previous startup’s blog (archive link here). Reposting it here since back then it was under an anonymous handle.]

My life has largely been defined by long shots. Definition of a long shot -

“an attempt or guess that has only the slightest chance of succeeding or being accurate.”

A few big ones which have affected my life -

Looking at my fascination with computers, my parents got me a computer way back in 1996 in India. They spent majority of their annual income to get me that PC. That one thing has defined the entire trajectory of my professional career.

Went to Kota to study for IIT. Got into one of the better training classes. Hated it, went into depression because of the shitiness of that place and eventually quit.

Got into one of the better undergrad engineering schools by giving a semi serious shot to one of the entrance exams (AIEEE) which back then wasn’t considered relevant by most students in the state of Gujarat.

A big one (grad school) - applied to Cornell, Berkeley and Stanford among others, even though only 1 person from my school had ever been accepted in any of those universities. I had applied to 11 grad schools in total, the above 3 were long shots, 4 more were medium chance, 4 more were backup. I got rejected from every single one, apart from Cornell. And Cornell was the last one to get back with a response - those 2 months of rejections were excruciating. I had given up on going to grad school when I received the acceptance email.

Chose to work in a startup at below market rate in the valley rather than a big company after graduating from school. Most of my classmates had starting salaries of 15% - 30% more than I did - but I got stock options. ;)

Started startup #1 - worked on it for one and half years - went nowhere and eventually shut down.

The current (and the biggest so far) one - left my comfy job in the valley, moved back to India, started a health tech company.

This is just the professional/career long shots. There are also ones in my personal life like making friends with people who were deemed “spoilt” and “bad influence” at some point or another. Telling a girl that I like her - while feeling very scared about how she will react… and so on.

Professional long shots

I want to focus on the professional long shots since many people don’t have problems taking them in personal life. I have no idea why - maybe because one has monetary consequences and thus makes it “more painful” than the other kind. But I digress.

I observed that there are a lot of people who would do well if they took long shots. I asked them why they did not do so. The responses largely fell into one of two categories -

I am happy where I am with a stable job and don’t want to take any risk or stress of doing things which have a large chance of failure.

I want to do it but I can’t. Because of X.

Choice #1 is personal and I don’t think anyone has the right to question how somebody chooses to live their life.

Choice #2 is the more interesting one. The X in the statement is in a very broad range. Its an array of infinite reasons such as -

I cannot risk a stable income - sole breadwinner of the family.

My family doesn’t allow me to do it.

I don’t think it will work because everyone I know has tried, failed and went through tremendous hardships afterwards.

Too scared to make the leap.

I tried before and I failed - it takes too much effort to do it, my former partners/cofounders screwed me over, investors screwed me over.

… and so on.

Note that there will not be one overarching reason which encapsulates X, but a combination of several of disparate reasons that come together to prevent them from taking a long shot.

The Privilege of standing on shoulders of others

As I thought about it, I came to the realisation that my “ability “ to take long shots wasn’t an ability at all. Most of it is due to an amalgamation of my circumstances - which started right from my birth.

- My parents also valued education greatly and went to great lengths make sure I was never held back in that area. They encouraged me to read books and bought any book that I wanted, even at the cost of other things in their life.

I don’t have any dependents that I need to financially support.

I have savings from my previous job which gives a good personal runway to have an extended period of time with little/no income.

I am a Software Engineer, and I know I am good at it - this gives me great comfort since I know that I can always fallback to having a job which is in high demand and has high pay.

My extended family has always been supportive of me trying new risky things. I have several role models (both friends and family) in real life who run their own businesses successfully.

There are several more such reasons. Every single one adds a pillar of support for the platform on which I can attempt risky endeavours, and as I have progressed in my life the number of pillars have only increased.

These pillars give me “The Privilege” of taking long shots. I am standing on shoulders of people in my life who support me unconditionally.

A lot of people cannot or will not rightly choose to take long shots because they have very few or no support to take them in the first place.

You will always hear examples of people taking long shots who had no pillars at all and made it big. But just like startups, for every one loud success there are hundreds if not thousands of silent failures.

I salute those people, the ones who did not have “The Privilege” but went ahead with the long shot anyway. You have more heart and courage to face the shit that life throws at you than I ever will.

Original post here.

Note - Some parts of this post requires some familiarity with basics of Machine Learning. These terms should be understood in a technical form - Binary Classification, Accuracy, AUC, Biased Dataset

TLDR -

It is possible to build a system to predict medical chronic conditions with ML.

The path to actually building an ML system from scratch is long and winding and will probably differ for each company. It’s very messy and undefined in the start - and that's ok.

You do NOT need a huge amount of data to start. Use proxies and try to get some real world data points.

There are plenty of ways to use the initial results in a commercial application - especially for targeting/identifying customer types.

Read the newly released Machine Learning Yearning if you don’t know where to start.

DoctorC has been doing diagnostics for 3 years now. We have gone from running operations on spreadsheet to end to end automated systems which handle everything, from placing an order, to delivery of reports. We also store all reports and some report values for our customers so that they can easily access their medical history from anywhere.

A few months ago, we started an initiative to see if we can apply ML on our data and get some useful applications out of it. We are an extremely small company with very limited amount of resources. So it was basically just me running some experiments and dedicating only a few hours a week.

The first question we asked ourselves before starting was - “How is it going to be useful to us as a business?”. The answer was surprisingly easy.

We have two kinds of customers -

People who undergo some illness/accident etc where they need some medical help - but only one time.

People who have chronic conditions like hypertension, diabetes, cholesterol issues etc.

#1 are transactional and #2 are our repeat customers. #2 use/need our service the most and they are consequently very valuable to us. In Urban India, chronic conditions (non communicable diseases) are on the rise rapidly and the prevalence of diseases vary from 15% all the way to 35% (WHO link - http://www.who.int/choice/publications/Chronic_diseaseIndia.pdf).

These are the customers we are looking for. So the question boiled down to - “How do we identify people with chronic conditions?”. Once we identify them - we can enroll them in a program specifically tailored for people who need medications and diagnostic tests frequently.

The hardest point - Starting

We needed to start with a training set of people who are identified as chronic. We had 0 points of data for this.

So we looked for proxies. “What behaviour indicates if a person has a chronic condition?”. Answer - People who used our services frequently are more likely to be chronic.

We keep track of every test that was bought and their results for every customer. Therefore, we had a (comparatively speaking) rich medical history at our fingertips. On top of that - our in-house proprietary system can “read” a diagnostic report and infer structured medical data for a whole array of tests.

Based on this, we started with an initial training set of a couple of hundred people who were deemed “chronic”. We used another set of infrequent buyers as negative examples for our model.

I explored a whole bunch of options on what platform to use for training ML - Scikit Learn, Tensorflow, AWS Machine Learning and Microsoft Azure. Azure’s ease of use puts it in another league altogether - especially for people who want to just apply existing ML algorithms. Drag and drop, kicking off experiments parallely, easy to construct flows etc. allow you to move really really fast as compared to writing code.

This might become a bottleneck later as we scale or it might become more expensive to run - but the tools they provide save tons of time, especially for prototyping and experimentation. I highly recommend trying out Azure - their free tier gives a good taste of what’s possible on their platform.

Before you start, or while you are playing with your dataset - you need to decide what your “One True Metric™” is. This is the thing you are going to optimize for while evaluating different machine learning models.  By definition, it is very specific to your problem and cannot be purely decided based on a formula.

For us, it was identifying people with chronic conditions with a highly imbalanced data set (we have a lot less people with chronic conditions compared to normal population). It’s a binary classification problem. I chose AUC as a measure and started experiments to optimize that. Why did I not choose Accuracy? That is a post for another day.

We did an initial split of 50/50 of positive and negative samples to ensure we didn’t bias the system with a small initial set. Then i started my experiments. This is the messy part - you have to start from the simplest models and run different models simultaneously on the same data.

The first set of results yielded AUC curve similar to this -

Tight Iterations

We did our first real world test with a model of an AUC of 0.58 (which is not a very good score). We took the top 100 predictions by the model and started verifying if the identified people were chronic. We recorded this data and then used it to create a newer model. This gave us another 100 predictions. Then we repeated the cycle again.

The model started improving with each iteration. Reconstructing new model with new data took only a couple of days and the model training process itself was only a few minutes. Gathering the data was the only bottleneck.

Our model’s curve improved as shown below. And our total dataset from a raw data standpoint was around 1.2 million discrete values. Our accuracy was approaching ~80% (first couple of hundred predictions for every iteration). The AUC of the best model would vary from 0.74 to 0.80.  The accuracy had a steep fall afterwards, but it was still significantly better than the 15% to 20% accuracy you would get with a random sample.

Predictive Factors

More interesting were the predictive factors for chronic conditions. While I cannot delve into too many details, one of the most interesting find was that Vitamin D (both the number of purchases done and the values of it) were one of the most predictive factors for someone being chronic or not. It neatly fits into the current research which suggests that Vit D is extremely important to maintain since it affects almost every system in our body. You should get Vit D levels checked every 6 months or a year - especially if a large part of your day is spent indoors.

Conclusion

Yes, you can predict, with a high degree of accuracy as well as high AUC, if someone is suffering from a chronic condition based on a combination of their purchase behavior (surprising) and their test values (not very surprising, still exciting). The next question was whether we could actually hone down to a specific condition like Diabetes, Thyroid disorders etc. That is a post for another day.

There is also the long term possibility of actually providing diagnosis while working side by side with doctors. This system is a small step in fulfilling that vision. We have miles to go before we sleep.

In a startup, there are always 100 different things that need to be done at any given point of time. There are umpteen ways you can accomplish those tasks. Many of them can be solved by using tools that other companies provide. Some of them clearly require something to be custom built internally in your organization.

And then there are some, often pesky, things that are in the middle. You can solve the problem by using some third party tool/software, or you can also build a replica of that tool internally with a few tweaks here and there that would be nice to have. Its these situations which cause a lot of sunk time in wasteful projects that could have easily been solved by just using some existing solution out there.

But how do you decide when its good to build something in-house vs just buying your way out of the problem? I would like to walk you through two situations where the build vs buy question typically pops up.

The first one is the problem of “Big data” (or even little data for that matter). The world is awash in data, but data is not the same as information. Data is just a fact, or an observation. It is information which empowers your organization to make decisions.

There are entire companies built to help you convert your data into actionable information. Whether it is to help you track your funnel for sales conversions, keep track of how many times in a user returns to use your product per day/week/month, at what points are your users getting frustrated and dropping off, and a million other things.

Building and maintaining such systems in-house is an expensive, time consuming affair. Adding to the difficulty is the fact that the people who are well versed in this field are rare, and are super hard to recruit. Therefore, there are a lot of companies out there who promise to provide you the same kind of insight into your data at a fraction of the cost and time of building in-house systems.

Many of the products currently on the market are pretty good at their specific use cases. New Relic, for example, is an excellent way to monitor your infrastructure and the raw performance of your application. Google Analytics is another great free solution to measure the basic metrics of your website / mobile app.

But, its not as well defined when it comes to business metrics - especially if you are a startup. A startup by definition is something new and its business model is undefined early in its life. This means that *nobody* - including the founders themselves - know what the best way to measure the business is. This means there is no one stop shop which will have all the things that your startup needs to measure its health.

As an example, at my current company - DoctorC, we needed a dashboard which measured all our critical metrics for the current day at a glance. We wanted to know how well each of the city was doing and what was contributing to its progress as well as how far or close we were to our daily goal overall. It should also update in near real-time so we know whats happening now as opposed to 60 mins later.

There is simply no solution out there which would provide such functionality out of the box or that can be modified like this without significant tech development. And since it is so critical to our business we built the dashboard from scratch. A screenshot of the dashboard, with fake numbers, is below -

Each bar would turn green if the goal of each acquisition channel is hit. The circle for the city would also turn green once its goal passed. The bar on top would similarly green once our overall daily goal was hit. The more green we saw the better our business was doing - we can process everything in a very visual manner. It also acts as motivational tool where the entire company would try together to get as many greens as possible every day.

This investment has paid off immensely. This is exactly what is right for our organization specifically, not for anyone else. It has helped us move fast in decisions since we have the right data at our fingertips. It was an explicit, conscious choice we made to strengthen our business.

What we did not do - we did not build our own infrastructure monitoring tool - we use New Relic, we did not build our a/b testing/optimization tool - we use Optimizely, we did not build our own servers - we use Amazon Web Services, we did not build our web analytics software - we use Google Analytics and Heap Analytics.

We use off the shelf software as long as they are saving us time and money and are very well defined in nature. Anything that is specific for our business and which we absolutely must have our way - without any compromises - is built in-house.

The second example of an in house tool that is core to a large business is New York Times’ internal Content Management System (CMS) called “Scoop”. It powers all their workflows for publishing articles on the web, mobile and print. Apart from that, it allows them to render images that are automatically cropped and aspect ratios adjusted so that the writers and editors don’t have to worry about managing how images are displayed. There are several other features which are very specific to NY Times such as “Story Budgeting and Planning”, a tool used to coordinate the publication of articles across multiple news desks across the world in different timezones in a painless way. Its tightly integrated into their workflow in a way that is not found in a generic CMS.

For NY Times, the entire experience of writing, copy editing, publishing and post publication editing is handled seamlessly with minimal friction with Scoop. They are currently changing “Scoop” so that they became a digital first organization. They have this flexibility because they have built Scoop from the ground up for themselves. It is beholden to no other organization than NY Times itself. The results speak for themselves, nytimes.com and its mobile site are industry leading media websites in the world that improves constantly day after day after day.

Anything that you find unsatisfactory in the market and doesn’t serve your business needs is up for grabs. After that, its all a question of prioritizing what you want to build. Another point to note is that the “in-house tool” does not need to be fancy or use heavy technology. It can be as simple as a simple spreadsheet which is filled manually every day - we still do that for some of our internal projects.

In the end, there are 2 simple things that you must ask yourself -

Is the thing you want to do critical (and i mean really absolutely “Oh my god, my business will die without this!” critical)?

Are there zero off the shelf tools that will allow you to do *exactly* that critical thing?

If the answer to both is yes, you should seriously consider building it in-house, even if you just started your startup yesterday. This philosophy is, rightly so, independent of the size of the organization.

Note - This article appeared in an edited version in Business World here.

Image credits -

Big Data

TL;DR -

A client side model is not equal to a server side model.

Therefore, a REST call made by the client can involve several objects of different types on the server.

Remove the thought that there is a need for one-to-one mapping between an object of client side and server side. Once it is decoupled your design becomes simpler.

Earlier in the year, I was going trawling through the internet looking on how best to design a REST API for DoctorC.

Doing a toy project, I implemented a CRUD view in Django (our backend) which updated an “Author” object and a “Book” object. I moved onto implementing REST for a real life scenario which involved creating and updating orders in our system.

Image credit - http://chez-syl.fr/2013/12/les-requetes-rest-en-php-get-post-put-delete/

I straightaway ran into problems. Booking an order involved -

Updating the customer’s object.

Optionally creating a couple of other types of objects.

Sending a confirmation email and SMS.

A bunch of other things.

The logic was obviously a little complicated, the creation of objects is supposed to be done in a certain order. On top of that, and depending on the input if something failed midway, then we need to rollback the entire flow of creation of objects.

I knew that doing this on the client side is gonna be a nightmare. It involved broadly -

Creating and maintaining all these different models/objects on the client.

Make distinct server calls to save each of them.

Rolling back everything if something fails in between.

Looking at the above it immediately made sense that having transactions on the server side would take care of all of it. But, of course, if I was implementing a REST API, how on earth can I have a server side transaction for creating and/or updating different objects?

And thats when the solution popped into my head.

All the REST API tutorials I have come across make an unmentioned assumption that a model/class on the client perfectly maps to another model/class on the server. I had carried that along while trying to get it working on a real life project. If I let go of that, everything started making beautiful sense. A “ClientOrder” on the client can contain all the data and just pass it to the server via a /api/order POST call. On the server all the data from “ClientOrder” can be unpacked and the corresponding “Order”, “Appointment”, “Lead” etc. objects can be created on the database. We can also support transactions in a trivial manner because its just a single API call.

In hindsight, this looks stupidly obvious. I had to give it a lot of thought because the assumption of one-to-one mapping of ClientModel <—> ServerModel is very subtle and no one explicitly mentions it in their REST tutorials.

Ok. So we recently closed our seed round at DoctorC.

Since this was the first time ever we had raised money successfully, a lot of “Congratulations!” were rolling in. Its supposed to be something to celebrate. Right? But all I felt at the end of it was… uneasiness.

Uneasiness at the fact that a bunch of really smart people have given you their hard earned money to go build an awesome company. You start to doubt and ask yourself - What if i fail? What if the things we believe as a company are untrue? What if we screw up in some unimaginably stupidly way which seems obvious in retrospect and could have totally been avoided? - and so on and so forth. The more I thought about it, the more the terror surfaced of disappointing the very people who have taken the risk and made a bet on you just on your word that you can deliver. Now not only are you answerable to yourself, but you have the responsibility to spend other people’s money wisely. This terror has the capability to consume you, give you cold sweat, sleepless nights endlessly - if you overthink about it.

But.

I realized, there is another way to think about it too. Every investor understands the risk and reward of investing in a startup. They know how fragile a startup is and all the ways it can go wrong. They also know the kind of reward they would get if the startup succeeds. The statistics speak for themselves - 90% of startups fail outright, while 5% perform modestly and a very tiny percentage actually hit the ball out of the park. They know the odds as much as I do - and they have made an informed choice to take the risk with you.

Of course, the terror never goes away, it just becomes much more manageable by not dramatizing the whole thing inside your head as a default.

I continuously remind myself of this piece of advice Ram gave me -

“In a startup, the highs are never as high as you think nor are the lows as low as you think. Do not get caught up in the moment.”

At a small startup like ours we like to celebrate our successes. Earlier in the year, when we were just getting started, the number of orders we used to get were few and far in-between. We wanted to know when an order was booked - as soon as it happened. This was a problem.

Our office is a converted bungalow, its split across 2 levels and has rooms on each floor. Thus, we did not have open space where people knew what was going on with others easily. Typing on google chat for a sales person every time they booked an order was a tedious task and since we were following Spreadsheet Driven Development, we did not have an automated system of tracking it. (Yes, our entire company was running on a spreadsheet when it started.)

So Neehar came up with an idea of getting an old school autorickshaw horn to “announce” that an order was booked. It was prevalent in the 1990s but now, its rare to see that kind of horn. It has a very retro feel to it. Its amazing fun to press the air filled bulb and have a happy loud noise come out on the other end. Sneaking up on someone and playing it right beside them soon became a favorite prank for everyone in the office.

As soon as we started using the horn, someone started answering the horn by shouting “Woot!” in return. Before we knew it, it became a tradition to hear the horn, and celebrating it by “wooting” in return. It became a reflex action for us.

This inevitably meant that the office was getting noisier as we started scaling up. Since we were in a residential area, we decided to stop using the horn so as to not be a PITA to our neighbors.

It had a visible effect on people. It was almost eerily silent. We did not know how well the day was going unless we continuously checked our dashboard, and the general level of fun in the office plummeted. We weren’t as pumped about sales happening and our general morale was at a lower level. As a short term fix, we created a hook into our company wide slack channel to announce when an order was booked in real time.

It made things slightly better, but not as good as it was earlier. We needed our horn back, but in a way which did not annoy our neighbors by being too loud. So we hatched up a plan to use web socket connection to our server from our internal dashboard and use it to make the sound right in the browser.

We had a working solution out in 24 hours and it worked brilliantly. Since its implementation, the office has gone back to being fun and every time we hear the air horn, few people would inevitably woot and elicit smiles and laughter. The more horns we hear, the happier we get. It also became more personal since we can hear the air horn wherever we are, not just in the office. I once wooted in reflex action while sitting alone in my bedroom.

At DoctorC we like to move fast, but not break things all the time. To do this while maintaining a break neck speed of a startup, we came up with a methodology which we use to experiment with new features. Neehar christened it SDD (Spreadsheet Driven Development). We kinda arrived at it while following one of the tenets of the gospel of Paul Graham - “Do things that don’t scale.”

Here is how it works -

Someone in the company pitches an idea for a new feature.

As with every other idea in a startup, we aren’t sure whether its gonna be useful or not.

The quickest way to experiment is to do it - manually.

Create a spreadsheet (a google doc so its easier for collaboration) which basically behaves if the feature was developed completely. Use it with live data. More often than not its just mocking what a couple of tables on the database would look like. The business logic is just a bunch of humans updating it by hand every time.

Test and iterate on it. Tweak it to fit your needs.

Decide whether you want to roll it out with full support from dev team. If not, then keep the spreadsheet as an archive of your record of your experiment.

If yes, then forward the spreadsheet to the dev team and they use it as the design doc for the feature. No requirements doc needed.

Profit

I want to add that the manual part is really painful and slow. It makes people who have to do the work complain, grumble and generally not be in that good of a mood. But the benefits of it are phenomenal. We can change our experiment on the fly, find out what is useful and what is not. It enables us to have a solid view of what the feature should look like from end to end. This keeps the dev team focused and moving fast.

Most of our experiments run for 2 weeks before we make a decision. And since we run more than one in parallel it helps us test out a lot of stuff without bringing everything else to a screeching halt.

Its been 5 months since I made the hardest (so far) decision of my life.

In June 2014, I moved to India to work on a startup.

On the surface it doesn't sound that hard. But I want to back up a bit to set the context. I had moved to US from India in Aug 2010 to do my master's at Cornell. After that, in 2011, I joined a gaming startup Funzio. It was an amazing experience where I grew a lot. We got acquired by Gree in May 2012 and I was working at Gree ever since then.

So within 3 years I had moved from India, graduated from a damn fine university, moved to the heart of the tech industry, worked in a successful startup which got acquired, and was earning really good money for someone 2 years into their career.

But (there is always one "but" eh?) I always wanted to do a startup on my own. From my undergrad days I used to gorge on techcrunch for tech news and was fascinated by startups and entrepreneurship. Then I discovered Paul Graham and Hacker News. It lit a fire inside me. Here was finally something where you can learn about all things tech and a community which actively pursues entrepreneurship.

From May 2012, during nights and weekends I was working on the now defunct startup called Shoutt. In early 2014, I managed to hop through the hoops of USCIS to work on it fulltime, but by that time Shoutt was on its last legs. It wasn't too long before it was clear that Shoutt was gonna die.

Mansi and Neehar, my co founders of Shoutt, had moved to India in 2013 to work full time on it since they could not stay in the US due to the visa requirements. Around the time that we decided that we needed to move on from Shoutt, they came across another idea - DoctorC - which seemed a pretty good one to pursue. They wanted me to join them if possible. There was an option where I could work on it while living in US but it made absolutely no sense to me to do that when the product, the market, the people and the business was in India.

Winding down Shoutt was gonna take about a couple of months and I needed to decide whether I wanted to work on DoctorC or find a new job in the Bay Area.

Several people helped me reach the decision and I wanted to share the things that affected my thinking the most.

1. Being comfortable taking risks is fundamental to the journey of entrepreneurship

One of my mentors (an entrepreneur himself) had mentioned this to me when we were talking about the life of an entrepreneur. You have to be comfortable with the fact that there are several unknowns and risks in your path. When the risks are very real and staring at you in your face, this attitude keeps me from going down a rabbit hole of what-ifs to insanity.

2. Don't let immigration issues ever be a major factor in your long term career decisions

The incredible lawyer who secured my visa for Shoutt gave me this advice when we were talking about the implications for me if I went back to India. This had an oddly calming effect on me, because the visa issue was the biggest thing that concerned me at that time. He made me realize that giving up my H1B is not an irreversible decision, I have enough of a network in the Bay Area that I will not "lose my roots" from here.

3. It will be not that hard to get back up if I fail completely and utterly

This ties in with point #1. Failing is part and parcel of everyone's life (more so for an entrepreneur). The world will not end if DoctorC fails completely. In fact, it will be a unique experience which I will carry with me for the rest of my life.

4. The opportunity was too damn good to pass up

While talking about Shoutt to one of my mentors (who is a serial entrepreneur), I mentioned DoctorC and the opportunity to work on it. His reaction was - "Stop wasting your time with Shoutt. You need to start working on this immediately." This was a couple of months before we actually decided to shut down Shoutt. Those words struck a chord within me and are looping over in my head ever since.

5. Regret Minimization Framework by Jeff Bezos

I don't know where I came across this. For me, this is clearly an extremely good technique to make any long term decisions easier. Its advice by Jeff Bezos which he dubs "Regret Minimization Framework". Summarizing here -

Project yourself to when you are age 80 and you are looking back on your life. I want to minimize the number of regrets I have. Now if I decide X and it turns out to be a failure, would I regret doing it when I am 80?

I applied the same logic and the answer was immediately obvious to me. And it has held up so far. I don't regret moving to India to work on DoctorC.

6. The pros of living in India

My parents have been incredibly supportive through this transition. I can visit them more often, good indian food (though I still love the bay area more for its variety) etc. Plus, I wanted to learn golf since a while, the entire 10 lesson course costs less than $100! :)

My time capsule randomly died on me with a power failure. It would not turn on. Since Apple is such a reliable company and known for good customer service, I expected that they'll have a look at it, try to fix it and if it cannot be saved then get my data out of it.

After talking with their service I was told that they do not service time capsule. Period. It doesn't matter what the issue is, they wont open it. It is a completely ridiculous policy and totally un-Applesque. The customer support told me to leave feedback here: http://apple.com/feedback - and here is what I wrote -  

Hi There,

I had a chat with your support group earlier and they told me to leave the feedback here.

I own a time capsule, iPhone, iPad, and a Macbook. I buy apple products knowing they are of high quality and in case of failure, good service and customer support.

Unfortunately i am sorely disappointed by your policy of not servicing time capsule in case of failure. My time capsule died and would not power on. Apple refused to open it and service it saying its "against your policy". My data is stuck in an nonfunctioning device which is not even looked at by the company which sold it to me in the first place.

This is close to the level of "comcast customer service" and I am aghast to find out that Apple follows this kind of policy. Please fix it.

This article is on top of HN right now. It delves into how Google is bringing competition into one of the most entrenched part of tech industry in America.

As I was reading it one point really stood out as a bit... meh. The relevant point was this

These companies have a point. Even Grande CEO Murphy admits that most consumers don't need to go that fast. He added that even if they subscribe to such a service, the equipment and devices in the home aren't capable of delivering those full speeds. Few customers even subscribed to the company's highest tier of service, which previously topped out at 100Mbps, before it introduced the 1Gbps service.

David Noonan, who covers broadband for consultancy IBB, said that most families couldn't consume enough online media to justify a 1Gbps connection.

I disagree. For this I point you to Alan Kay's video - The Future Doesn't Have To Be Incremental. One of the central points of his talk was as follows -

Imagine the technology 20 - 30 years from now (Faster than 1Gbps speeds IMO are expected at that time)

Along that timeline, project it backwards to how it will look about a decade from now.

That technology 10 years into the future cutting edge stuff is already available today... but its just incredibly expensive to get hold of it.

Put a bunch of smart people with this futuristic technology in the same building and let them play with the possibilities.

According to the video, creating Object Oriented Programming and GUI wasn't possible with the "current" technology at that time. It was only because they had access to these incredible technologies which were about 10 years into the future that they could imagine and experiment the possible uses of it. As he is famously quoted - "The best way to predict the future is to invent it."

Google is doing the same with GoogleX. And its trying to do it at a much larger scale with Google Fiber. Instead of just a few scientists having access to "the future" they are allowing everyone to have it. Of course people "...couldn't consume enough online media" to justify a 1 Gbps connection because nobody made the assumption that everyone will have access to those kind of speeds. Online streaming video would have never been possible with 56k dialup modems either.

Different companies have different “culture”. Everyone knows what it feels but I haven’t come across anyone who has quite put a finger on what the term “company culture” encompasses.

I am a freshly minted co founder and this thought has been nagging me for a while. I wanted to have some sort of non nebulous definition of it so that I can understand it. Once I understood it, I can actually work on moulding it to what my co founders and I expect from others in the company and with each other.

I was talking with my co founders about what expectations we had for X. And it struck me then - the company culture was what other people in the company expect.

There are several levels of expectations and each of them contributes to some part of the so called overall “culture” of the company. The expectations like -

Responsibility - What sort of responsibilty do you expect from your co founder? Your employees? Your peers? Fresh Grads? Senior level employees? Do you see a developer just devloping features and then moving on. Or does she support the feature post production too? Who tests the product? What kind of testing is deemed enough to push to production? What about on call duty? Do you have a dedicated person on call? Or do you rotate between the members of the team? How do you define a “team leader” and what sort of deliverables do you expect from her?

Working hours - How often do you want people to burn the midnight oil? Do you want the company to provide bunk beds? Or do you have a strict clock in clock out policy as it is mandatory for people to take breaks? Do you want have an unlimited vacation policy? Are you ok with people taking unfair advantage of it? How often do you schedule “fun outings” for the team? Is 1 day every 2 weeks too often or too less?

Transparency - Should every employee of the company know the financial health of the company? Or only managers? Should you allow employees to have company source code on their personal laptop? How do you handle/express disagreement in the team?

Freedom - What about allowing people to use the office resources for their personal projects? Who owns the IP from the company hackathons? Do you think its inappropriate to reddit at work? Where do you draw the line?

Shit storm policy - What do you do when everything goes wrong in a project? What is the process to handle “fuck ups”. Do you call out the person who messed up. How do you make sure it doesn’t happen again? When do you decide its a problem with the person and not the process? How do you express that its not acceptable to do X but without personally attacking the person who did X.

I have barely scratched the surface of the stuff that actually encompasses everything that defines the culture. I have phrased all of these as questions because it is very important to explicitly ask yourself everytime anything happens in the company, especially if you are in an influential position. It will define how your day to day interactions work with your team/company.

I am part of the generation of programmers where Stackoverflow and Google were already alive and thriving when I first started programming seriously. My coding workflow has Google tightly integrated into its loop. The primary reason is I don’t need to remember the exact details of implementing… anything.

Here is what a (totally made up trivial) typical flow looks like -

Hmmmm. I have these user ids in a dict as keys, but I need a subset too based some criteria X. I also need to get the remaining users who are not part of the subset.

Ok, this seems a good use case for sets. Lemme look up on how to do it in < insert-your-implementation-language >

And then I google “sets in < implementation-language >” and off I go.

This usually happens when I haven’t touched a particular language for a while (a few weeks).

I don’t remember programming syntax anymore. I just look it up and then forget. Short term holding. Its not costly anymore to forget, and IMHO, this is more of a blessing than a curse. Assuming that you have access to internet while programming is very valid. And having a vast trove of information at your fingertips all the time changes everything.

Google has become a layer of abstraction for me. I think of the problems purely in terms of application logic/algorithms and then try to figure out how to implement it using the chosen tool… and there is absolutely no cost associated to switch back and forth between the two. I just throw away the implementation stuff once I have used it, because its so easy to look up later.

Many times, I am not a programmer but an abstract systems thinker where I mostly think in terms of the domain of the problem rather than the bare bones technical spec. This is a good thing since I get clarity in understanding the product and dont get too bogged down in the technicalities. I don’t know how far it is from being an architecture astronaut but i keep myself grounded by implementing the solution too.

I was catching up with friends while vacationing in India and I found out that one of them had quit his job a few months ago and was “… figuring out what to do next.” I was very surprised to hear that he had made that decision. He was absolutely brilliant at his job and was progressing quite nicely in his career.

He said -

I was doing well and I felt that I was getting in a comfortable position… you know I finally had a relatively smooth routine at work where I knew what I was doing and I had learnt a lot over the last couple of years… and that was a sign that its time to move and change. Because I don’t think now is the time I should be getting comfortable. So whenever I feel that I am getting comfortable, I freak out.

This statement struck home to me. I always had a similar feeling where if I was getting “too comfortable”, I felt that there was something missing in my work. I should also point out that both of us are still in the early stages of our career (both of us only have a few years of experience). This point is even more so applicable to people like us than, lets say, a 50 year old who is supporting their 2 kids' college education.

The quote above succintly laid out a principle which will guide me atleast for the next few years, and hopefully, my entire life. So whenever I start feeling comfortable, I know that I need to start doing something different.

I recently read Scott Adam’s Book How to Fail at Almost Everything and Still Win Big (Affiliate Link) . Its a very different book in terms of the style of writing and how it presents the “life lessons” that the author learnt. I would highly recommend going through it just to have a feel of the way its written if not for anything else.

One of the more striking things that resonated with me was to “be selfish” in a good way. Its different from the way Ayn Rand describes it - which most people interpret to behave like assholes - but I digress.

Scott Adams says thus -

When it comes to generosity, there are three kinds of people in the world:

Selfish

Stupid

Burden on others

That’s the entire list. Your best option is to be selfish, because being stupid or a burden on society won’t help anyone. Society hopes you will handle your selfishness with some grace and compassion. If you do selfishness right, you automatically become a net benefit to society. Successful people generally don’t burden the world. Corporate raiders, overpaid CEOs, and tyrannical dictators are the exceptions.

…

The most important form of selfishness involves spending time on your fitness, eating right, pursuing your career, and still spending quality time with your family and friends. If you neglect your health or your career, you slip into the second category - stupid - which is a short slide to becoming a burden on society.

…

Generous people take care of their own needs first. In fact, doing so is a moral necessity. The world needs you at your best.

…

Being selfish doesn’t mean being a sociopath. It just means you take the long view of things.

The above view clarified a lot of what I kinda sorta had stumbled into and felt like the correct thing in my head. I just couldn’t express it as clearly.

I unconsciously behaved in a similar manner. When I was a student, I barely had any money to look after myself. I used to count every single dollar I spent and had none to spare for charity.

I graduated, the first thing I did was to pay off my debt - all of it. At the end of my first year, I barely had 1k in savings, but I was debt free. The money kept on coming in, I naturally saved a decent amount of “rainy day money”, sent a bit of it back home in India and started investing the rest. I had a car, a really nice laptop… and I quickly ran out of things that I actually needed. Everything else I did/bought from this point onwards was a luxury.

It was as if a switch went on inside of me at this point. What do I do with money that I have to spare? How do I make the world a better place? The easiest answer to this was to donate it to whatever cause I felt was good. I had a problem though, many charities use their donation money for “the overhead cost” and I was skeptical of finding one that guaranteed to donate 100% of the money to the needy. Fortunately, Watsi launched at the same time and my problem was solved.

Scott Adam’s experience was also similar -

Once all of my personal needs were met, my thoughts automatically turned to how I could make the world a better place. Apparently humans are wired to take care of their own needs first, then family, tribe, country, and the world.

Yep. I was just being human. This behavior is also exhibited by the super rich like Warren Buffet and Bill Gates with The Giving Pledge where the amount was $125 billion as of 2010.

Have you ever wondered why flight safety instructions explicitly tell you to put your oxygen mask on first, before helping others? Why is this an important rule for ensuring survival? Because if you run out of oxygen, you can’t help anyone else with their oxygen mask.

I don't remember exactly where or when, but I came across a fact which boggled my mind.

Why were whales nearly hunted nearly to extinction? Cars.

Sperm whale oil was particularly useful as an automatic transmission fluid and US was importing around 55 million gallons of oil each year in the 1970s.

After knowing the fact though, I had a realization that I should've guessed that there is always an economic incentive which led to such wide-scale hunting of the whales. Indeed, there were several of them.

As of today, Jojoba oil is used as a substitute for sperm oil, but it was known since *1933* that jojoba contained oil that had similar properties to whale oil. But it wasn't used commercially until the late 1970s. So what changed in 1970?

Whale oil use was banned by the enactment of Endangered Species Act in 1973. Up until then, jojoba was considered "economically unsustainable" to be used as a whale oil substitute. But we now know that that isn't true, the automotive industry figured out a way to do it once their hand was forced by artificially making whale hunting more economically unsustainable. Those "pesky conservationists" succeeded in passing a law that forced the hand of the companies to start looking for an alternative.

The thing that left me with a strong impression is the fact that a good substitute was known for over 40 years (40 goddamn years!!!!) before it was actually used commercially. It led to lot of implications in general about the role of politics, technology, industry and how they interact and influence each other.

My first thought was on the incentivization of the commercial industry to innovate to look for better ways to do something just because its "better for humanity". Looking at history and our track record, this is simply a naive outlook on how corporate companies operate. Their sole existence is to make (profitably) what the consumer wants - environment, humanity etc be damned. Time and again its proved with Blood Diamonds, Financial Collapse , Sperm Whale Oil... and the list goes on. Anyone who claims that the free market will solve it because "...once the consumers know what the implications of buying a product are, they wont and the demand will dry up..." is in a pipe dream.

This leads to the conclusion that you need some sort of regulation, especially for things/people who have little to no voice (literally in the case of whales) in positions of power. A government's job is to ensure that it looks at much larger issues than just what is "economically profitable" and that we do not end up in a direction that kills our species and the planet. In the case of "Endangered Species Act", it worked wonderfully for the whales. It challenged the status quo and nudged technology in the right direction - two things essential for good innovation.

Technology is a wonderful tool to improve the daily lives of millions of people. But the organizations/individuals that create them often have some circumstances or a reason that prevents them from taking the "right path". And after deliberating on it, there is never a good enough reason. Whenever YOU, as a creator, are making something, think about the impact your product is gonna have on the world. Is it worth it? Are there any alternatives out there? Will I feel proud of what I am doing on my death bed? Does it even really matter? There are no clear cut answers to many of these, especially when one is creating something new and nobody in the world has seen, let alone used it. But these questions should be a framework and a mould which guides you in your craft.

Finally, as consumers, we need to do our bit and research on what sort of impact we have on this planet and rest of humanity. It takes a lot of work to do so and that's why I do not believe that this approach will ever reach a critical mass where every consumer in the market is aware about every product they are consuming. It is simply too much for one human being. Very few people thought of whales while buying cars... and very few people are aware about it even today. Thus, it falls back on the government, non profits and those pesky conservationists to be on the lookout for the rest of us.

I will end it with another fact. As recently as 2008, NASA (of all the organizations) [used a solid white wax called Spermaceti] [3] as a lubricant for the machines like Hubble Telescope in space.

Since it doesn't freeze in sub-zero temperatures, spermaceti was used in Nasa's space missions – no substitute could be found for this natural lubricant.

Will it be another 40 years before we find out that an artificial substitute could have been used all along?