This question has haunted me incessantly on and off over the years.
Out of those harrowing years pondering the perplexity of programs, one answer stands out:
Software is difficult… because of people.
Not to be misconstrued as an apathetic, people-cynical take. Quite the contrary, I think every person has value and depth deserving of their own spotlight.
That said, there’s this perception that software development is naturally complex, which is true to a degree! Tricking rocks into thinking is a dark art with many decades of forbidden knowledge and required tributes powering it. Certain problems have multitudes of layers to them: hard performance requirements, regulatory compliance, distributed processing and communications, translation into rock language, etc, etc.
But writing code tends to be more complex than strictly required; that’s because software is created by people for people, with each party bringing their own expectations and perspectives on things. These expectations rarely align, and the perspectives are frequently narrow.
It’s at these crossroads, the conflicts between people, where code becomes its most complex — undermining another common misconception that software is purely logical, apolitical, and objective. Code is none of those things. People write code, and bring with them all of their ideas about the world.
Software development is a social performance, with all of the bureaucracy and bargaining that comes with that.
Difficulty Selection
So what, exactly, do people even DO to make coding harder? Who are these villains? What’re their tragic backstories?
Well, you see, it’s all of us. We’re the Real monsters.
*cough* …
Honestly though, the issue of overcomplexity is a mixed bag of mistakes, the human condition, ego, and priorities. It’s something we all contribute to, many without realizing it.
Our code interfaces grow grotesque, documentation falls into neglect, and tooling shrivels out of existence.
Casual Mode: The Easy Stuff
It’s unavoidable - we make mistakes, we misunderstand, make assumptions, and generally have flaws.
Nobody’s perfect.
Our individual contributions to software complexity are innumerable (very much including my own).
It is incredibly common to approach a problem with a preconceived notion of how it should be solved — the patterns and names involved, how people will use your solution, etc. Those notions rarely, if ever, cover the whole spectrum of the problem and its many needs, leading to situations where a developer assumes that something will only be used one way, but the consumers of their code need to use it another - or use something in a way it wasn’t intended to be used because it’s not clear HOW it should be used. It’s also common to name things in a way that makes sense to one’s self, but no one else can grep what those names mean.
We’re also human, with various personal faults and situational limitations.
Sometimes we misinterpret something, sometimes we (read: I) don’t want to put too much effort into figuring something out. Certain situations make it difficult to dedicate resources to learning a library too, so our use of it hinges on what we can glean at a glance — of the documentation, ideally, if it exists. On that note, making documentation is a time and resource sink that’s not always easy to justify given our constraints (or ability to motivate ourselves to do it).
Technical communication is a whole study in and of itself which we can’t all be masters of. With, more often than not, shaky communication skills, misunderstandings are bound to happen. While a better technical communication emphasis within programming circles would be incredible, conveying its importance to our employers escapes reach.
All of these things are the norm; you’ll encounter these issues in any scenario where people have a hand in its construction (especially me). They will never be fully eliminated, but we can take steps to minimize their impact through our APIs. More on this later~ (Foreshadowing~ I’m using that word right, right?)
Normal Mode: The Defaults
Beyond the human psyche exists another level of misfortune weaving: its foreboding visage taunts us at every turn through the corporate world.
Business Priorities.
Software complexity feeds on the schedules of crunched, time-limited developers under pressure from their management.
Our time is expensive. In some ill-conceived cases, our lines of code are too. Money makes the capitalistic world turn, alas, so making the most of what time we have (or other “genius” limiting metric) takes priority. Unfortunately, that may mean taking shortcuts to get something in on-time, or de-prioritizing other things like code usability / maintainability. It’s difficult to convince the Suits™ that money spent here would show long-term gains, especially when we can’t always guarantee as much due to the nature of having ever-growing complexity in our projects.
The Suits™ Have Their Own Priorities
Time to put towards code hygiene and tooling proves fleeting when it lacks that marketable pizzazz. That viral flair. That… Je ne sais quoi.
Something like, say… Machine Learning rebranded as the more appealing “AI” — the current hot trend, which is why we see so many companies ham-fistedly chasing it.
Large feature trend chasing like that takes resources away from quality of life (QoL) work both user-side and developer-side. Trends never let up, the marketing machine chugs along unendingly, so eking out little code and experience improvements between deliverables is about the best we can manage, if even that.
Sometimes, We Too Must Don the Suit®
Whether it be working on a personal project or communal open-source project, we have our limits. Especially when you’re moonlighting on top of being a full-time developer.
Faced with overwhelming adversity, something has to give — it’s usually the code usability, woe upon us. Even if the project is explicitly an API of some sort, cracks form along its surface from a lack of ability to really prioritize the interface’s experience within our target feature velocity. It’s the same story with documentation and tooling; things not directly feature-related fall by the wayside.
We may also have to ensure certain thresholds of stability, be it in the end product or our code interface. Usually, it’s to ensure operability for others who depend on our code. There’s a trade-off for that stability, and it may be for the retention of bad API boilerplate and bottlenecks.
These issues all add up over time, becoming tech debt and an overbearing maintenance overhead. If we could get our world to slow down, to give us the resources to resolve what we need to, these problems could be eliminated. Until then, we make do with what we have.
Although, there may be an evil beyond even this amongst the mortal realm…
Hard Mode: Git Good
Reaching beyond the beyond until we’ve circled back to humanity, a symphonic choir begins to sing over top an epic, aggressive score. Everything seems normal, and yet… The music, it plays.
They’re in our midst. Them — the apathetic, the selfish, and the toxic. Those to whom making software difficult to write is a goal.
Part of human society is dealing with Them, truth be told. But that doesn’t mean we have to like it. Or never discuss it.
To fend off the pedants, I should point out that all of these traits exist on a spectrum: we all exude them in varying doses. Lower doses fall into the Casual Mode section above as things we all do passively. There’s a threshold where it becomes a problematic pattern of behavior, however, and that’s what I want to draw attention to.
Programming appears to draw those who have an exceptional sense of superiority for being an expert in a complex domain. Some see the complexity as a requirement, others as a barrier to entry that ensures their status and recognition. In any case, there are a few who insist on maintaining the current (or a more rigorous) difficulty threshold. Obviously, this keeps programming difficult by its very intent.
To be fair, many may not even realize they’ve fallen into this mindset. They may have legitimately ingrained into themselves that certain complex workflows and mindsets that work for them must work for everyone who is a “good” coder. There’s also an element of falling for the “this is how it has always been” mindset, or internalizing these processes and complex interfaces as a gauntlet to prove one’s worth because it was an ordeal they (the person continuing the practices) had to get through. Those aren’t great reasons still.
Which Takes Us Nicely to the Next Group - The Apathetic.
Those who don’t care what chaos their code (or actions) cause. Their underlying motive? I don’t know, I’m not a paleontologist (Pathologist?).
They may not have ulterior motives, but quite frankly could not care less about programmers with less domain knowledge than them. Including less knowledge about specific libraries, approaches, and mathematics at large. Their contributions to the wider complexities of thinking rocks cannot be overstated, especially when they hide behind communal calls to “assume good faith”. Not to belittle the need to assume good faith in large communities prone to miscommunications and full of people who aren’t professional technical communicators — but it’s also worth analyzing how it can be a drain on code communities and contributions.
Some even see complexity as an inherent good worth pursuing for its own sake. You see this with systems that, admittedly, are complex, but whose developers actively refuse to provide any sane defaults or easy routes / tooling for new users / programmers (*cough*Vulkan*cough*). Certain OS communities are also known for being this way; to avoid having to go into witness protection, they shall go unnamed. To be balanced, the other OS communities are quickly gaining ground here too. Much to my chagrin.
And Then, There are the Overt Problems.
To not put too fine a point on it, there are those who are just malicious, for whatever reasons.
Those who just want things their way or the highway. People who can’t take critique, are generally abrasive to outside views, or take back routes to avoid receiving opinions as much as possible. Programmers that will abuse processes and procedures to ensure their changes get in despite any raised concerns. Or those who know they can out-endure others in a community by being as loud and repetitive with their view as possible against all opposition.
There are people who hold themselves in high regard; people who have a vision for the project / a new competing standard that is only theirs, is narrowly considered, and results in a worse interface for everyone else. Wanting to tackle a new approach to an existing bad interface is A-ok, even approaching a library beyond our current capabilities, but being conceited and toxic about that idea — belittling the original interface’s devs, or pushing a project into disrepair for a poorly conceived replacement without taking input / having safe transition plans / leaving afterwards — goes too far.
Again, it should be said that these things exist on a spectrum — having a little bit of these traits doesn’t make you abusive or toxic. There’s a threshold you have to cross to get to that point. But gee, don’t a good number of programmers like to cross it on the regular?
Incantation to Lift the Curse?
In just three easy steps. Continue reading to find out how!
…
That’s it. That’s the incantation. It’s very misleading, I know.
For real, while there are steps we can take to improve our tooling, documentation, and code interfaces to make software easier to develop overall, there’s no panacea. Anyone trying to tell you there’s some magic cure-all is selling you snake oil and headlight fluid.
It’s very much worth noting that I’m an individual developer with my own experiences and biases, and I am in no position to create some grand plan of action for everyone. These come entirely from my experience, mixed in with my takes on the experiences of other software engineers I know.
First, acknowledge the problems that cause software development to become difficult in the first place. Our passive individual contributions, priorities, apathy, envy, malice, etc.
From there, determine what we can do for each problem within our own means. As individuals, it’s (relatively) easy to work on the personal aspects — to catch ourselves taking shortcuts, skipping the hard parts of learning something new, avoiding tedium. Patricia Aas has given a few talks on how to approach new code bases.
Learn good API design. Check out the thoughts of programmers like Kate Gregory, Jason Turner, Herb Sutter, and similar developers who have a focus on the usability of code. Make interfaces that are easy to use, difficult to misuse, and self-documenting (as much as possible). Provide shortcut ways to accomplish complex tasks with sane defaults (while still providing full-control when desired).
Develop better tooling. Automate the annoying things, don’t leave the project setup to a long list of prerequisites and assumptions, cover the rough cases (like prerequisites), and ensure any annoyingly complex processes have a tool. Consider a GUI. Check the first time setup periodically — using system containers on cloud platforms (like Docker or Kubernetes). Get that first time setup down to as few steps as possible. Something like: IDE installation, library download, run “Setup” - build and done! Maybe remove any unique considerations for your project, and lean into existing tooling to handle the setup.
Beyond that, we can take steps to fix our own companies by advocating for ourselves. Sadly, companies may not listen, no matter how much buy-in and strong motivations with tangible outcomes we provide. That’s when we Unionize! All we can do is try to get buy-in by sheer number of supporters / contributors for our proposal, or seek better employment where they allow for a healthy development environment.
One specific thing worth advocating for with our companies is the use of, and ability to contribute to, open source projects. Open source is not perfect, and it is rife with many of the aforementioned specters, but it’s a way to create a better programming ecosystem available to all. One of the main problems with open source is getting active, professional backing behind it for any meaningful amount of time. Professionals who want to contribute frequently have to do so in their already shrinkingly small free time. If we can convince our companies of the benefits of not only using open source (easy sell, depending on the license and if it fits our needs), but also contributing during company time, it would be an overall boon to the community. Again, if they say no, Unions are right there…
Last, but not least, keep an eye out for toxic contributors to projects. Resolve those situations with as little noise and drama as possible — elevate only where needed. People who are being toxic may not realize what they’re doing, and deserve chances to improve.
All of those pieces combined might slowly get us to a world free of this curse of overly-complex software development. (At least, from my limited perspective.)
I honestly don’t know why I chose such a complex first real post topic. This ended up being way more in-depth than originally conceived. Apologies if parts of this come across as rushed - they almost certainly were. ;_;