Software is process
19 July 2024
When I was younger, I wanted to be an inventor. From Doraemon to Kiteretsu, I dreamed of cool gadgets that could make my life easier. I’ve spent months playing with DC motors and magnets to build useless gadgets, but those were the joys of childhood. Eventually, I got a computer, and since then, I have spent most of my time “building” software, hobby-wise and professionally. I think of myself as an engineer, and for the longest time, I’ve felt that as engineers, our place in society is to build stuff. We are supposed to arrive at the scene, analyse the situation, and present a solution.
Recently, I’ve realised that the solution is not always a tool that is to be built. Instead, the solution is often a process that is established. I wonder if this is why it makes sense for countless engineers to become consultants whose job is to establish processes and recommend tools for others. Some consultants may participate in the tool-building process, too, but that is not the prevalent image of their profession anyway.
Thus, engineering is both about building tools and establishing processes. Excellent, good summary, but this is a cheap argument. The more interesting point is that all engineering does impose a process. A tool does not exist without context. The simplest tool imposes a process. Using a hand-held hammer to mint coins requires a metalworking process that can be managed with the manual strength of a blacksmith. The process has a complex relationship with the tool. It does not dictate the tool but guides it. If you discover new materials or want to be creative with your coin, you might have to invent new tools.
I request your attention to the following conclusion. First, for most engineering professions, process design has evolved to be done by different people than tool builders. For example, a door has to open “in” or “out,” but that’s for the architect to say. The mechanical engineer must only concern themselves with the hinges and how they creak. There is a relationship in the arts, but that is a relationship between two professionals.
This is true for buildings and architectures, factories, mechanical engineers and industrial designers, automobile engineers and car designers, etc. Thus, in retrospect, always in retrospect, I could have anticipated that product managers would play such a prevalent role in the software industry a decade ago. Not so long ago, everyone was a full-stack developer in software. The tool builders were the UX designers. That was all before JavaScript released its hellspawns upon our world. Hold on, the conflation is still there for “non-important” stuff. Similarly, a skilled civil engineer might find designing a vanilla 2 BHK apartment easy enough. But for everything “important”, such as Meta and its apocalypse horsemen, you get an army of product designers, owners and other folks whose job is establishing processes, not the actual building. A more striking example would be data teams in large enterprises. Databases are perfect examples of tools in a tight loop with the read-write process. You need an entirely different infrastructure for something that requires small but frequent read-writes compared to infrequent but large read-writes. Databases are an exciting engineering problem because a one-size-fits-all solution is mathematically impossible.
Given this background, I find it shocking that nothing equivalent to building codes is prevalent in software. Non-sarcastically “important” software such as tax filing systems, healthcare, and travel systems all rely on software that could have bugs. Still, no agencies regulate the basic standards required for software to be in the market. Your identity can be stolen. You can be denied access to critical services. (People in India get denied food rations due to issues with the biometric authorisation systems). Software systems impose a political will by virtue of being a process. They are a power which must have checks and balances in a democracy. Computers are a great technical innovation. They can make enforcement better. If you are a non-binary person and you must choose a gender between Male or Female on an internet portal to book a flight, what do you do? Perhaps if you were booking a flight with a human cashier, they would simply pay no attention to a few “incorrect” fields on the form. But that would be inefficient. That would be “bad data” due to “human ineffeciencies” and that’s precisely why we got the computers to replace the stupid humans! The AI world has had more conversations around “ethical AI” and regulating the “LLM giants”, but nothing has been done. Even if they come up with something, that’s not the beginning or the end. AI has more negative PR, thanks to Arnold Schwarzenegger and thanks to Eliezer Yudkowsky??
Be aware, be vocal. Do not accept lousy software. Push for higher standards (of software) in your life.
Coincidnetally, this was published during the week of the Crowdstrike outage.