Infinite Undo

I think that if the mental model one is using is like the New View on Systems Safety, then the connections between the disciplines in TOAD (Testing, Observability And Devops 🐸) naturally become obvious as you work more and more with software systems.

Conversely for people rooted in the traditional / Cartesian view of software systems, it’s pretty much impossible to explain. (edited)

For instance

If I think that the history of a software is composed of discrete, finite frames of time that can be reconstructed after the fact then good fucking luck explaining to me why I need an observability framework that lets me respond quickly to unknowns.

Because if I think history is composed of discrete, comprehensible frames that can be reconstructed after the fact, then I have a lot to learn from the system out of that history — your observability and prediction techniques are nice but not necessary.

However

If I don’t accept that history is composed of discrete comprehensible events, then I have very little to learn from history and then the o11y framework and the culture of resiliency makes sense This is btw what is obliquely being gotten at in the Clay Shirkey quote:

Process is an embedded reaction to prior stupidity.

It means people with a lot of process are people with the old / Cartesian view of system safety. They have a lot of process because they think they can predict what sort of events will be important in the future.

Since formal process is reactive, it follows that in order to have a lot of formal process, we have to have a lot of events we can predict with great specificity.

Phenotypic conformance analysis describes the practice of establishing a static analysis ruleset based on the observed properties of the codebase rather than on an existing open source standard or a ruleset voted upon by the team.

How do you know what you know?

stuff you can test

How Change Works In Large Organizations

The Kübler-Ross change curve and the Six Phases Of A Project are two time-tested ways of visualizing how organizations cope with change!

Here the Kübler-Ross curve and the Six Phases are together for the first time!

If testing software and writing code feel very different to you, it’s only because you haven’t written enough code yet. That is only my own admittedly controversial opinion. I believe that everything we call “software testing” is a subset of the activity we call “programming.”

Implementation is a test of a hypothesis. To implement a pattern in code one must first form a narrative or if you prefer a hypothesis. Implementation is itself a test of whether the narrative holds up.

Corollary: Consider that the full specification for a program and the program itself are the same thing. This implies you can’t design computer programs by up-front, complete specification. You are constrained by “the laws of nature” to begin with an incomplete hypothesis and proceed by testing the implementation of said hypothesis, using the results of that test to decide how to go about modifying either the hypothesis or the implementation or both, then repeating that process.

At all levels of the stack, it is always the case that complete specification is functionally impossible since such a thing could only exist in the form of a complete implementation. The largest Web application and the derpiest hello world program both have this quality: that they cannot be implemented by complete specification but must be built iteratively via (in)validation of hypotheses.

Thanks everyone for upvoting my post on HN!

It's always exciting to see my work on the front page of Hacker News :)

Software rot

The software development life cycle is predictable in that any long-lived product will eventually outgrow some of its subsystems. For instance a Web service that begins life with a single monolithic database server will as it scales need the capacity increase that comes from a distributed database. A historical example can be found in Twitter’s original dependence on the ActiveRecord ORM, which over time was replaced with a variety of databases and services.

For historical reasons

In one sense this might be considered as a canonical definition of legacy systems: the system contains subsystems that are not optimally suited to day-to-day functioning despite the fact that at some point in the past those same subsystems did function optimally. There is a concept of Software Rot or Bit Rot that metaphorically encapsulates the life cycle phases that precede this sort of legacy system.

Frictionless yet it still wears out

The observed course of the software development life cycle is that features begin life in a “working” state — meaning that they satisfy the requirements agreed upon by an empowered group of stakeholders — but that inevitably the same features begin to exhibit bugs that are not in any way related to changes of code nor hosting environment. Software rot (as this phenomenon has come to be called) is caused because the requirements for features continue to change and evolve even after such features are in the hands of their users.

the system contains subsystems that are not optimally suited to day-to-day functioning

This is not a well-understood area of software production, nor does Computer Science have much to contribute by way of solutions. The problems are not algorithmic but environmental, social, aesthetic — in other words it is what programmers like to call a squishy problem because so much of the problem space is taken up not by software but by humans and their co-collaborators.

In programming, soft skill is hard

The idea of engaging with squishy problems is uncomfortable to a lot of programmers. I think this is because programmers currently do not have an opportunity to learn the heuristics that would allow them to distinguish good solutions from bad when it comes to human-and-social issues.

Taking software rot as an example, it is a well-known phenomenon long documented in the literature of software. Yet it has no commonly-agreed up on solution. Its management is not a topic of discussion in job interviews nor in performance evaluations (for the most part). The contraventions for software rot are not listed in general programming books nor taught in coding boot camps.

I do not believe that software rot is ignored as a topic because no one recognizes its importance. I’m pretty sure it’s ignored because no one feels comfortable giving advice about it, because almost no one has successfully dealt with the long-term requirements-changes and subsystem upgrades that go with solving software rot.

The knowledge about how this problem have been successfully dealt with are locked away in a couple of books. And those books are old, using server-side programming and Java as their example environment. That’s a hard sell to a junior engineer fresh out of a boot camp or undergrad program.

Suboptimization is THE reason for technical debt. But I have to go outside #programming to find discussion of it. https://t.co/cgTN57MrT4 pic.twitter.com/SKjYKUVV4O

The uncomfortable truth is that dev is a complex relationship with an increasingly intelligent other.

The uncomfortable truth is that dev is a complex relationship with an increasingly intelligent other.

Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON.

Here's how I use Miller to pipe CSV data into jq

jq is currently my tool of choice when it comes to processing all sorts of data. Except XML and CSV. XML is pretty well handled by xmlstarlet but I have never found a CSV parsing tool that I liked. I just make do with using jq to work with CSV and to be honest I find the results I can achieve to be substandard.

Anyway, Miller solves all that and is easy to install even if you don't use homebrew and need to compile it from source.

mlr --c2j cat my_file.csv | jq .

"Testing vs. Checking"

"Testing vs. Checking"

You will find this meme under discussion at all of the leading software quality assurance (SQA) conferences today including GTAC, STPCon, STAR, CAST and the various Selenium confs.

In short the discussion revolves around whether a Cartesian divide can be effected between the human and the computer's role in software quality assurance.

"Testing vs. Checking"

It is not the discussion itself that bothers me. Although Cartesian division is out-of-fashion with Devops Luminaries(tm) such as myself, I can certainly see the utility of a Cartesian point of view when establishing certain foundational local taxa --- as we Test Architects often must do!!!

Concern trolling

What jerks my fucking chain though is the god damned concern trolling.

The biggest problem by far is all the fucking language policing that goes on in the testing and in the agile community. Language policing is microagressions. Stop it and apologize.

The most common form of Language Policing (aka Tone Policing) that I've seen take place around "testing vs. checking" is "correcting" people who choose to identify as automators of test activities.

I've personally been interrupted while speaking at a conference at least three times only to have the interruptor explain that "checking isn't testing." Yeah motherfucker. I already knew that and I don't give a shit about it becasue from my perspective that's minutia that everyone takes as written in the first place.

Software testing considered as a series of band-pass filters

Good systems converge on quality. They snap back into shape when deformed. Autoscaling is an example of how Web systems regain their good state after some event has impacted functionality.

Bad systems act the same. when pushed toward a better state they gradually edge back toward the bad state. Take for example the familiar case of a legacy system that has been put under test but gradually edges back toward untestedness again.

Teleonomy confirms what we intuitively already know. That some systems are easy to collaborate with and that other systems seem adversarial in our interactions with them.

The principle of Teleonomy shows that intent is not required in order for a system to pursue a goal. Neither collaboration nor antagonism requires intent.

Alexander is very clear in defining QWAN: Good systems pursue the goal of inner consistency. Bad systems pursue a state of internal conflict. when humans pursue a goal of internal consistency then we are pursuing the same goal as a well-functioning complex system. That the goals are aligned is sufficient to say that human and computer are collaborating. Intent is not required: the observed state of alignment qualifies fully as collaboration.

The "snapping back into shape" of complex systems is a practical example of system memory, one of the defining characteristics of a complex System as opposed to a simple one. Memory is also a characteristic of cellular automata such as the Abelian sandpile model.

So computers can be seen (without resorting to metaphor) as collaborating agents that not only have goals for the future but also remember the past. This is the observed reality of working on the computer system: it is difficult not to eventually regard computer systems as having motivations and goals which do not always align with our own. Teleonomy and sympoiesis are lenses through which this observed reality makes sense.

Dijkstra once said that the metaphor of "the bug that crept in while the programmer wasn't looking" is intellectually dishonest; because any bug in the system must have originally been put there by a programmer. But Teleonomy implies sympoiesis between human and program and that in turn calls Dijkstra's statement into question.