#computer-programming

To start, I fundamentally disagree with his politics. His distrustful attitude towards structure and process that makes things easily predictable and repeatable. He seems to miss the importance of communication and good managers (they exist!) when working in large teams. His focus on the tooling, ability, and culture seriously understates the importance of project structure. The product vision and importance of the team as a larger part of the organization has a massive impact on how the software gets 'put together'. The process of actually making something that works as quickly and as cheaply as possible isn't something to be derided, it's the basis for industry that eventually brought us to the information age. He often weaves his political views into his technical topics, and it's important to separate them.

M O. C clearly has an appreciation for elegant infrastructure, putting innate value in it's complexity and the ability to manage it. There is a high degree of skill required to build the architectural masterpieces: expressive languages, libraries, subsystems. Those are often the 'fun' problems posed in computer science, but I think he misses the problems found in 75% (This is generous) of businesses. Building good business systems can be a much tougher task than building the foundational blocks, because you have to combine these two disparate fields with conceptional models. The complexity then isn't only computer science complexity, but people, domain, time, and budget complexity. Not everything can be as hot as AI, but it can be much more complicated.

It comes down to how the sausage gets made. These problems have to be solved by people, with what they know, in the time and constraints they have. No one wants to make a big ball of mud, but they happen for very legitimate reasons. People are learning how to not make all the same mistakes of the past, but when things get built there will be old problems mixed with the new. M O. C prides himself as a software craftsman, with an ability that would qualify as art if code were prose. M O. C's most insightful posts were showing how ugly the sausage really was and hammering all of the ways to end up in bits. He nailed many of the poor justifications for common mistakes and why the world of software works the way it does. It's great treatise in failure, and even he laments the lack of a better solution.

I could count the number of pages that showed I spent too much time reading and mapping out code when I made much bigger steps in the debugger. I could see that my tests were effective because the projects where I wrote tests first had fewer issues later when I made changes. The length and detail of my notes was a metric I used to measure my technical progress.

But for my most recent project, I found that metric wasn't informative anymore. I took very few notes during most of my tasks (user stories), and a disproportionate amount of my notes were on one or two really nasty infrastructure bugs that I needed my team to help fix. I found I wasn't taking notes for myself anymore, I was taking notes to share what I was doing with my team. This was great for stand-ups and other collaboration opportunities; I always had an updated list of the specific technical problem I was on and what other related problems were in progress or just solved.

Upon review, I found them to be mundane and uninformative. Noting that I was 'testing and integrating fields i,j,k from the database to the front-end' was important for my team if they had issues accessing or verifying the correctness of field k, but now that information is irrelevant. I don't have any information about what I was thinking in a technical sense outside of my commit history.

I'm now going to start keeping my domain notes separate from a technical journal. I didn't like the experience of looking back at 6 month project and thinking "Did I actually learn anything?". I knew I was getting things done, my commit history and those fancy graphs of story points proves when and where that happened. I now need to keep track of what articles I'm reading and what tools I'm learning to use. Before, I didn't realize that over the course of the project I gained a whole ton of T-SQL experience until a team member asked how in the heck I knew how to do a feature that needed it.

I don't know if I'll post my insights here, as I feel they might be trivial things like learned more tmux shortcuts or used an sql output clause. Or even less useful notes about internal tool quirks and infrastructure experiences.

When senior engineers have a postmortem on a serious issue, I know I learn about system design, communication, and failure assumptions. These are great lessons I can't quite appreciate because I don't make these kind design decisions on a daily basis, but I still cherish the knowledge for later use and I know it will make me a better engineer to just accumulate those high level lessons.

A programming state of mind

Most of these lessons aren't things that are relevant while actually writing code. They are stories to guide my hand as I make smaller design decisions. If you think of coding as brick laying, it's important to think that every small decision will eventually bubble up the highest level. Simple things like paying attention where your variables are being allocated, and if you are passing copies or references can end up having a performance impact that may change how the system operates as a whole.

Laying out your thoughts before you start coding is mostly a function of what you are keeping in mind. Do you need to be juggling all of the different states of your object? Do you need to be thinking about race conditions and thread interactions? It would be impossible to keep every lesson in mind for every piece of code, so having the optimal memory layout is extremely important before setting out to do anything.

Where is the line?

At what point have you thought enough about a problem to be ready to tackle it? The macro and micro scales have different thresholds. On the small end, having prototyping projects can be great when you don't know where you are going. Your prototype should run into walls and roadblocks you weren't aware existed, those become lessons to keep in mind when writing the project. On the macro scale, when are you ready to be in charge of your own project? If you only ever worked on enhancing and rebuilding things, does that help you when you start from scratch? There isn't a prototyping stage for a job; most companies don't let you leave for a month to try something else before deciding to come back or not. Deciding that you are ready to strike out on your own can be a much bigger commitment than spending a day trying a new technology stack.