Ignoring YAGNI – 12 years later

Fourteen years ago, we started to build a distributed system to gather environmental data in an automated 24/7 fashion. Our development process was agile and made heavy use of short iterations (at least that was what they were then, today they are normal-sized). So the system grew with many small new features and improvements, giving the customer immediate business value.

One part of the system was the task scheduler. Because the system had to run 24/7 and be mostly independent of human interaction, the task scheduler’s job was to launch different measurement processes at the right time. We had done extensive domain crunching and figured out that all tasks follow a rigid time regime like “start every 10 minutes” or “start every hour”, regardless of the processes’ runtime. This made the scheduler rather easy to develop. You should keep it simple, after all.

But another result of the domain crunching bothered us: The schedule of all tasks originated from the previous software system, built 30 years ago and definitely unfit for the modern software world. The schedules weren’t really rooted in the domain, they all had technical explanations like “the recording of the values is done sequentially and takes up to 8 minutes, we can’t record them more often than that”. For our project, the measurement hardware was changed, so our recording took a couple of milliseconds. We could store and display the values continuously, if the need arises.

So we discussed the required simpleness or complexity of the task scheduler with the customer and they seemed pleased with all the new possibilities. But they decided that the current schedules were sufficient and didn’t need to be changed. We could go ahead and build our simple task scheduler.

And this is when we decided to abandon KISS and make the task scheduler more powerful than needed. “But you ain’t going to need it!” was the enemy. Because we knew that the customer will inevitably come around and make use of their new possibilities. We knew that if we build the system with more complexity, we would be the heroes in a future time, wearing a smug smile and telling the customer: “We’ve already built this, you can use it right away”. Oh how glorious this prospect of the future shone! Just a few more thoughts going into the code and we’re set for a bright future.

Let me tell you a few details about the “few more thoughts” with the example of an “every hour” task schedule. Instead of hard-coding the schedule, we added a configuration file with a cron-like expression for the schedule. You could now leverage the power of cron expressions to design your schedule as you see fit. If you wanted to change the schedule from “every hour” to “every odd minute and when the pale moon rises”, you could do so. The task scheduler had to interpret the configuration file and make sure that tasks don’t pile up: If you schedule a task to run “every minute”, but it takes two minutes to process, you’ve essentially built a time-bomb for your system load. This must not be feasible.

But it doesn’t stop there. A lot of functionality, most of which wasn’t even present or outlined at the time of our decision, relies implicitly on that schedule. Two examples: There are manual operations that must not be performed during the execution of the task. The system goes into a “protected state” around the task execution. It disables these operations a few minutes before the scheduled execution and even some time afterwards. If you had a fixed schedule of “every hour”, you could even hard-code the protected timespan. With a possible dynamic schedule, you have to calculate your timespan based on the current schedule and warn your operator if it isn’t possible anymore to find a time slot to even perform the manual operation.
The second example is a functionality that supervises the completeness of the recorded data. The problem is: This functionality is on another computer (it’s a distributed system, remember?) that doesn’t know about the configuration files. To be able to scan the data archive and say “everything that should be there, is there”, the second computer needs to know about all the schedules of the first computers (there are many of them, recording their data on their own schedules and transferring it to the second computer). And if a schedule changes, the second computer needs to take the change into account and scan the data archive for two areas: one area with the old schedule and one area with the new schedule. Otherwise, there would be false alarms.

You can probably see that the one decision to make the task scheduler a little more complex and configurable as required had quite some impact on the complexity of other parts of the system. But this investment will be worth it as soon as the customer changes the schedule! The whole system is programmed, tested and documented to facilitate schedule changes. We are ready!

It’s been over twelve years since we wrote the first line of code for the more complex implementation (I’ve checked the source control logs). The customer hasn’t changed a single bit of the schedule yet. There are over twenty “first computers” and they all still run the same task schedule as initially planned. Our decision did nothing but to add accidental complexity to the system. It probably introduced some bugs along the way, too. It certainly increased our required level of awareness (“hurdle of understanding”) during the development of features that are somewhat coupled with the task schedule.

In short: It’s been a disaster. The smug smile we thought we’d wear has been replaced by a deep frown. Who wrote all that mess? And why? It wasn’t the customer, it was us. We will never be going to need it.

Domain-aligned bugs

Frank C. Müller [CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0) ]

Imagine that you are an user of a typical enterprise software that handles commercial products and their prices. There are different prices in the software that are somehow related to each other. There is the purchase price that indicates your cost if you buy the product. There is the retail price that gets listed in your price lists and is paid by your customers, should they buy the product. You probably already figured out that the retail price should never be lower than the purchase price, because that would mean you lose money with every successful sale.

Let’s say that the enterprise software not only handles products, but also parts. Several parts combined, with some manufacturing effort, result in a product. Each part has a purchase price, the resulting product has a retail price. The retail price of the product should be higher than the sum of purchase prices of the parts. If it isn’t, you lose the costs of the manufacturing effort and some extra money with every successful sale.

If for any reason you cannot clearly estimate your manufacturing effort, the enterprise software has another input field for an amount of money that you can add to the sum of the parts’ costs. We call this field the “sales bonus”. So, if you sell a product made up of parts, your customer has to pay a price that consists at least of the retail prices of the parts and the sales bonus. Of course, your customer has an individual discount percentage that needs to be subtracted from the total price. Are you still following?

You are now thinking in the domain of price determination and financial mathematics. If you were the user of said enterprise software, you’d probably expect some bugs like these:

  • It is possible to enter a retail price lower than the purchase price
  • The price of products manufactured from parts isn’t calculated correctly
  • It is possible to enter a negative sales bonus
  • The total price with discount could be lower than the sum of purchase prices of the parts without a warning

All of them are bugs in the domain. All of them can be explained to a domain expert or a user with terms and concepts from the domain.

But what about the bug when you sell a product that consists of three parts, each with a retail price of 10 €, and a sales bonus of 5 €. You want to create a quote for your customer and the price shows up as 34,99999999998 €. You are a bit bewildered and try to countervail the apparent rounding error by changing the sales bonus to 5,00000000002 €. After this change you get another crazy total price and your prices in the database are different from what you entered, too. Everything seems to destabilize and deviate further and further from clear cut prices.

As a programmer, you know what happened. You know what caused this effect of numerical instability. Somebody stored monetary values in a floating point number. You know that is a bad idea and you’d never do this. But this blog post isn’t about you or what you should do or not do. It is about the user, expert in his domain, that stumbles over the bug as described and has to make some decision on how to fix it. This user cannot use any knowledge from the domain to even understand the mechanics of the bug. You, as the programmer, cannot explain this bug in terms of things the user already knows. You need to be vague (“the software doesn’t store the exact values, just approximations”) or introduce additional complexity (“we store this value by splitting it into a significand and multiply it with a factor consisting of a fixed base and an exponent. We can omit the base and just store the significand and the exponent and express a very large numerical range in just a few bits. Think about how cool that is!”).

Read the last explanation again, from the viewpoint of a salesman. We want to add some prices in the range of a few €, slap a moderate discount on top and call it a day. We don’t care about bits or exponential formulas. That is not part of our domain and it shouldn’t affect our domain or software that works in our domain. Confronting us with technical details reflects negatively on your ability to solve our problems. You seem to burden us with your problems in exchange.

As domain experts, we want only domain-aligned bugs.

Book review: A Philosophy of Software Design

This blog entry is structured in two main parts: The prologue sets the tone, but may be irritating because it doesn’t talk about the book itself. If you get irritated or know the topic well enough to skip it, you can jump to the second part when I talk about the book. It is indicated by a TL;DR summary of the prologue.

Prologue

Imagine a world where the last 25 years of computer game development didn’t happen. A world where we get the power of 5 GHz octacore computers and 128 GB of RAM, but nobody thought about 3D graphics or interaction design. The graphics of computer games is so rudimentary, it consists of ASCII art and color. In this world, two brothers develop a game that simulates a whole fantasy world with all details, in three dimensions. The game is an instant blockbuster hit and spawns multiple cinematic adaptions.
This world never happened. The only thing that seems to be from this world is the game itself: Dwarf Fortress. An ASCII art sandbox simulation of a bunch of dwarves that dig into the (three-dimensional) mountains and inevitably discover the fun in magma. Dwarf Fortress is a game told by stories, not graphics. It burdens the player to micro-manage a whole settlement down to the individual sock – Yes, no plural. There are left socks and right socks and they are different entities with a different story. Dwarves can literally go mad because they miss their favorite left sock and you didn’t notice in time. And you have to control all aspects of the settlement not by direct order, but by giving hints and suggestions through an user interface that is a game of riddles on its own.
Dwarf Fortress is an impossible game. It seems so out of time and touch with current gaming reality that you can only shake your head on first contact. But, it is incredibly deep and well-designed and, most surprising, provides the kind player with endless fun. This game actually works!

TL;DR: Just because something seems odd at first contact doesn’t mean it cannot work. Go and play Dwarf Fortress!

The book

John Ousterhout is a professor teaching software design at the Stanford university and writes software for decades now. In 1988, he invented the Tcl programming language. He got a lot of awards, including the Grace Murray Hopper Award. You can say that he knows what he’s doing and what he talks about. In 2018, he wrote a book with the title “A Philosophy of Software Design”. This book is a peculiar gem besides titles with a similar topic.

Imagine a world where the last 20 years of software development books didn’t happen. One man creates software for his whole life and writes down his thoughts and insights, structured in tactical advices, strategic approaches and an overarching philosophy. He has to invent some new vocabulary to express his ideas. He talks about how he performs programming – and it is nothing like today’s mainstream. In fact, it is sometimes the exact opposite of today’s best practices. But, it is incredibly insightful and well-structured and, most surprising, provides the kind developer with endless fun. Okay, I admit, the latter part of the previous sentence was speculative.

This is a book that seems a bit out of touch with today’s mainstream doctrine – and that’s a good thing. The book begins by defining some vocabulary, like the notion of complexity or the concept of deepness. That is rare by itself, most books just use established words to deliver a message. If you think about the definitions, they will probably enrich your perception of software design. They enriched mine, and I talk about software design to students for nearly twenty years now.

The most obvious thing that is different from other books with similar content: Most other books talk about behaviours, best practices and advices. Then they throw a buch of prohibitions in the mix. This isn’t wrong, but it’s “just” anecdotal knowledge. It is your job as the reader to discern between things that may have worked in the past, but are outdated and things that will continue to work in the future. The real question is left unanswered: Why is it so?

“A Philosophy of Software Design” begins by answering the “why” question. If you want to build an hierarchy of book wisdom depth, this might serve as one:

  • Tactical wisdom: What should be done? Most beginner’s books work on this level. They show exactly what goes on, but go easy on the bigger questions.
  • Strategic wisdom: How should it be done? This is the level that the majority of good software design books work on. They give insights about your work ethics and principles you should abide by.
  • Philosphical wisdom: Why should it be done? The reviewed book begins on this level. It explains the aspects of software and sourcecode that work against human perception and understanding and shows ways to avoid or at least diminish those aspects.

The book doesn’t stay on the philosophical level for long and dives deep into the “how” and “what” areas later on. But it does so with the background of an established “why”. And that’s a great reminder that even if you disagree with a specific “what” (or “how”), you should think about the root cause of your disagreement, not just anecdotes.

The author and the book aren’t as out-of-touch with current software development reality as you might think. There is a whole chapter addressed to current “software trends” like agile development and unit tests. It has a total page count of six pages and doesn’t go into details. But it at least mentions the things it doesn’t talk about.

Conclusion

My biggest learning point from the book for my personal habits as a developer is to write more code comments in the way the book proposes. Yes, you’ve read that right. The book urges you to write more comments – but good ones. It talks about why you should write more comments. It gives you extensive guidelines as to how good comments are written and some examples what these comments look like. After two decades of “write more (unit) tests!”, the message of “write more comments!” is unique and noteworthy. Perhaps we can improve our tools to better support comments in the same way they improved support for tests in the last years.

Perhaps we cannot solve our problems with the sourcecode by writing more sourcecode (unit tests). Perhaps we need to rely on something different. I will give it a try.

You might want to give the book “A Philosophy of Software Design” a try. It’s worth your time and thoughts.

Zero, Maybe, One and Many

In object-oriented programming languages like Java, the compiler will improve its helpfulness if the application provides a rich type system or strong domain model. There is a whole field of study for type systems, called type theory, that is fascinating and helpful, but does not provide easy rules to follow for beginning software developers. This blog entry proposes a simple set of rules for a specific part of type systems (associations among types) that can be applied to a domain model as a rule of thumb. The resulting model will empower the compiler and the code completion of the IDE to help the developer with writing correct code.

Data knows data

Even the most basic domain models separate the data in multiple entities (often classes). For example, an employee class has an internal id, but knows about a person class and a salary class that are associated with this employee. This “knowing about” is modeled as a reference to a person object and a salary object. In this case, the reference is probably of the type “one”: The employee object knows about one person object and one salary object. This is the usual way to structure data.

If you learn about the UML notation of data models, you’ll see that associations (aka references between objects) are given great emphasis. There are several different kinds of associations that can be customized by multiplicities and such. It seems that knowing other data is a complex issue for types. It doesn’t have to be this way. Here are four ways of knowing other data that are sufficient for nearly every use case: Zero, Maybe, One and Many.

Four basic types of association

  • Zero: Knowing zero elements of something different is the usual default case: Your employee object probably doesn’t need to know about the payroll object of the company and therefore has no association to it. This means that there is no member variable of the type Payroll in the class Employee. No developer ever modeled a “zero” association by declaring a member variable and setting it to null. This would be ridiculous. We just omit the member variable and are done. Knowing zero elements of something is easy.
  • One: Yes, I’ve omitted Maybe at the moment. I’ll come back to it. Knowing one element of something is also not hard: You declare a member variable of the type, give it a good name (that’s the hard part!) and ensure that every instance of your class (every object) has a valid reference to an object of something’s type. If you call methods on this reference, you call methods on the object you know. As long as you live, the other object cannot disappear. Knowing one element of something is a long-lasting relationship.
  • Maybe: Sometimes, you want to know an element of something that isn’t there yet or you knew an element of something once, but it is gone. You know “maybe one” element of something. These associations are typically programmed in a cumbersome way by many developers. Instead of embedding the “maybe” aspect in the type system and giving the compiler a chance to help, it is burdened solely onto the developer’s shoulders by implementing the “maybe” like a “one” with the added possibility of a null reference if the element isn’t there. A direct result of this approach are null-checks in the code or NullPointerExceptions at places without such checks. One possibility to elevate the “maybeness” into the type system is to implement the association with a Maybe or Optional type. Instead of referencing a Salary directly that might be null if an employee isn’t salaried anymore, the Employee class references an Optional<Salary> object. This object might “contain” a salary or it might not. With a few adjustments to the conditional flow of the code, this distinction between “something is there” and “something is not there” doesn’t matter anymore. If the code is free from implicit Optional types (references that can be null), a whole category of bugs disappears and the code is freed from manually programmed type system checks. Probably knowing one element is the type of assocation that requires some thought and is often done on the wrong level.
  • Many: As soon as you want to know more than one element of something, you fall into the “many” category. Many-associations are not so easy to handle, because there are so many possibilities to express them. The basic types are arrays or lists. My recommendation is to use lists whenever feasible and only resort to arrays if it is necessary, because arrays are fixed-length and have the same problem of maybe-null-references: An array index might have been written yet or not. If you refrain from storing null references into lists, they express their filling level a lot clearer than arrays. And given advanced features like iterators, there isn’t even a need to ask for the filling level. An interesting observation is that the list-based many-association can also serve as a zero-, maybe- or one-association. It is possible to replace all other types of association with lists. You probably won’t want to do this, because with the maximization of multiplicity flexibility comes more complexity and reduced readability of the code. You should strive to minimize complexity. Only add many-associations if you really need them. Even just replacing a “maybe” (Optional) with a “many” (List) is a source of much unwanted code and uncertainty.

Advanced types of association

Of course, there are many more types of association that you’ll eventually need. A good example is the qualified association, often implemented by a Map/Dictionary that translates from the qualifier type to the qualified type. But they are rare in comparison to the four basic types.

Summary

If you get your basic associations right, your domain model will help your compiler and IDE to support and guide you. This is an upfront investment that pays off manyfold over the course of the project and eliminates the burden of attention to detail when it comes to accidental complexity like null pointers. Your project’s domain probably doesn’t contain null pointers, but the concepts of knowing zero, maybe, one and many.

The inevitable emergence of domain events

Even if you’ve read the original Domain Driven Design book by Eric Evans, you’ve probably still not heard about domain events (or DDD Domain Events), as he didn’t include them in the book. He talked about it a lot since then, for example in this talk in 2009 in the first 30 minutes.

Domain Events

In short, domain events are occurrences of “something that domain experts care about”. You should always be on the lookout for these events, because they are integral parts of the interface between the technical world and the domain world. In your source code, both worlds condense as the same things, so it isn’t easy (or downright impossible) to tell them apart. But if you are familiar with the concept of “pure fabrication”, you probably know that a single line of code can clearly belong to the technical fabric and still be relevant for the domain. Domain events are one possibility to separate the belongings again. But you have to listen to your domain experts, and they probably still don’t tell you the full story about what they care about.

Revealed by Refactoring

To underline my point, I want to tell you a story about a software project in a big organization. The software is already in production when my consulting job brings me into contact with the source code. We talked about a specific part of the code that screamed “pure fabrication” with just a few lines of domain code in between. Our goal was to refactor the code into two parts, one for the domain code and the other, bigger one for the technical part. In the technical part, some texts get logged into the logfile, like “item successfully written to the database” and “database connection closed”. It were clearly technical aspects of the code that got logged.

One of the texts had a spelling error in it and I reached out to correct it. A developer stopped me: “Don’t do that! They filter for that exact phrase.”. That surprised me. Nothing in the code indicated the relevance of that log statement, least of all the necessity of that typo. And I didn’t know who “they” were and that the logfiles got searched. So I asked a lot of questions and finally understood the situation:

Implicit Domain Events

The developers implemented the requirements of the domain experts as given in the specification documents. Nothing in there specified the exact text or even presence of logfile entries. But after the software was done and in production, the business side (including the domain experts) needed to know how many items were added to the system in a given period. And because they needed the information right away and couldn’t wait for the next development cycle, they contacted the operation department (that is separated from the development department) and got copies of the logfiles. They scanned the logfiles with some crude regular expression magic for the entries (like “item written to the database”) and got their result. The question was answered, the problem solved and the solution even worked a second time – and a third time, and so on. The one-time makeshift script was used permanently and repeatedly, in fact, it ran every hour and scanned for new items, because it became apparent that the business not only needed the statistics, but wanted to start a business process for each new item (like an editorial review of sorts) in a timely manner.

Pinned Code

Over the course of a few weeks, the purely technical logfile entry line in the source code got pinned and converted to a crucial domain requirement without any formal specification or even notification. Nothing in the source code hinted at the importance of this line or its typo. No test, no comment, no code structure. The line looked exactly the same as before, but suddenly played in another league. Every modification at this place or its surrounding code could hamper the business. Performing a well-intended refactoring could be seen as direct sabotage. The code was sacred, but in the unspoken kind. The code became a minefield.

Extracting the Domain Event

The whole hostage situation could be resolved by revealing the domain event and making it explicit. Let’s say that we model an “item added” domain event and post it in addition to the logfile entry. How we post it is up to the requirement or capabilities of the business department. An HTTP request to another service for every new item is a simple and viable solution. A line of text in a dedicated event log file would be another option. Even an e-mail sent to an human recipient could be discussed. Anything that separates the technical logfile from the business view on the system is better than forbidden code. After the separation, we can refactor the technical parts to our liking and still have tests in place that ensure that the domain event gets posted.

Domain Events are important

These domain events are important parts of your system, because they represent things (or actions) that the business cares about. Even if the business only remembers them after the fact, try to incorporate them in an explicit manner into your code. Not only will you be able to tell domain code and technical code apart easily, but you’ll also get this precious interface between business and tech. Make a list of all your domain events and you’ll know how your system is seen in the domain expert world. Everything else is more or less just details.

What story about implicit domain events comes to your mind? Tell us in a comment or write a blog entry about it. We want to hear from you!

How to approach big tasks

In the heart of software development lies “the system”. The system is always complicated enough that you cannot fully grasp it and it is built by stacking parts on top of another that are just a tad too big to be called simple. The life of a software developer is an ongoing series of isolated projects that are at the threshold of his or her capabilities. We call these projects “epics”, “stories” or just “issues”. The sum of these projects is a system.

Don’t get me wrong – there a tons of issues that just require an hour, a cup of coffee and a few lines of code. This is the green zone of software development. You cannot possibly fail these issues. If you require twice the time, it’s still way before lunchtime. And even if you fail them, a colleague will have your back.
I’m talking about those issues that appear on your to-do list and behave like roadblocks. You dread them from far away and you know that this isn’t smooth sailing for an hour, this will be tough work for several days. This isn’t just an issue, it is an issue by itself for you. You are definitely unsure if you can make it.

Typical small project management

How do you approach such a project? It isn’t an issue anymore, as soon as you get emotionally involved, it becomes a project. Even if your emotion is just dread or fear, it is still involvement. Even if your management style is evasion, it is still project management. Sure, you can reassign a few of these icebergs, but they will always be there. You need to learn to navigate and to tackle them. Hitting an iceberg in the “frontal collision”-style isn’t a good idea.

On closer inspection, every project consists of numerous parts that you already know a solution for – typical one-hour issues – and just a few parts that you cannot estimate because you don’t know how to even start. Many developers in this situation take the route of least resistance and start with the known pieces. It’s obvious, it feels good (you are making good progress, after all!) and it defers failure into an uncertain future (aka next work week). Right now, the project is under control and on its way. We can report 80% finished because we’ve done all the known parts. How hard can the unknown parts be anyway? Until they strike hard and wreck your estimates with “unforeseen challenges” and “sudden hardships”. At least this is what you tell your manager.

Risk first!

My preferred way to approach those projects is to reveal the whole map, to estimate all parts before I delve into the details. I already know most of the easy parts, but what about the unknown and/or hard parts? I don’t know their solution so I cannot reliably estimate their size. So I sit down and try to extract the core problem that I don’t know how to solve yet. This is the thing that prohibits an estimate. This is the white area on my map. This is the “here be dragons” area. If I spend my resources doing all the work other than this, I will succeed until I stand on the border of this area and see the dragon. And I will not have sufficient resources left. My allies (like my manager and colleagues) will grow weary. I will have to fight my hardest battle in the most inconvenient setting.

My approach is to take the risk upfront. Tackle the core problem and fail. Get up and tackle it again. Fail once more. And again. If you succeed with your task, the war is won. Your project will still require work, but it’s the easy kind of work (“just work”). You can estimate the remaining tasks and even if you’ve overspent in your first battle, you reliably know how much more resources you will need.

Fail fast

And if you don’t succeed? Well, then you know it with the least damage done. Your project will enter crisis mode, but in a position when there is still time and resources left. This is the concept of “fail fast”. To be able to fail fast, you need choose the “risk first” approach of task selection. To tackle the risk first, you need to be able to quantify the “risk” of your upcoming work.

Assessing risk

There are whole books about risk assessment that are interesting and helpful, but as a starter, you only need to listen to your stomach. If your stomach tells you that you are unsure about a specific part of your project, put that part on the “risky” list. If you don’t have a reliable stomach, try to estimate the part’s size. Do the estimation game with your colleagues. Planning poker, for example, is a great tool to uncover uncertainty because the estimates will differ. Just remember: Risk isn’t correlated with size. Just because a part is big doesn’t mean it is risky, too. Your crucial part can maybe be developed in an hour or two, given an inspiration and a cup of coffee.

Failing late means you’re out of options. Failing fast means you’ve eliminated an option and moved on.

Developer Experience means nothing

You’ve probably already heard about the concept of User Experience (UX) and the matching virtue of User Experience Design. If not, you might want to go check it out. I would suggest you start with the excellent book “The Design of Everyday Things” from Donald Norman. It has nothing to do with software development, so you won’t mix up User Experience Design with Graphical User Interface Design or even Graphics Design.

In short, User Experience is what an user of your software will experience while using your software. If your software makes them smile, you’re some kind of UX god. If your software makes them curse repeatedly, you’re probably not. You can try to improve the experiences of your users by making changes to the software. This is called User Experience Design and is an important part of software development. Most developers know nothing about User Experience Design.

What developers naturally know about is Developer Experience (DX), a concept that isn’t really defined in the literature. I made it up to explain my point to you. Every software developer has a favorite programming language and IDE. Remember, the source codes of the same problem in different programming languages will ultimately yield the same machine code. The machine is totally agnostic about your preferences for a certain programming language. Your choice of a programming language for a certain problem is more about your Developer Experience than anything else. Developer Experience is everything you feel and think during software development. A bad Developer Experience lets you swear about the tools, the code and everything and everybody else around you. A good Developer Experience gives you a sense of accomplishment and safety while coding and makes your work not harder than necessary.

Because you can’t read elsewhere about the concept of Developer Experience, I want to give two more examples and show how it affects the User Experience. The first example is a big, long-lasting software system that is in production and still developed further. If you have a software system in production, everything requires an additional thought about the matching upgrade strategy. You can’t just modify the database structure, you need to provide migrations. You can’t just add a configuration entry, you need to make it optional or consider the least harmful default value. In the project of our example, the developers had three possible approaches to a new configuration entry:

  • They could add it to the code but leave the configuration files unchanged. This required the code to handle the “absent from configuration” case in a useful manner. It required the developer to make effort in the code. The user would not know about the new configuration entry if not stated in some external documentation.
  • They could add it to the code and write a configuration migration script that added it to the existing configuration files on the customer’s installation. The code could now expect the entry to be present, but the developer had to write and test the migration script code. The user would see the new configuration entry with the default value.
  • They could introduce a new configuration file to the system and place the new configuration entry in it. The code could expect the entry to be present, because new configuration files were added to an existing installation during the upgrade process. The user would see the new configuration file and the new configuration entry with the default value.

You can probably guess which of the three options got used so excessively that users complained about the configuration being all over the place, in a myriad of little one-liner configuration files with ominous names. The developers chose the best option for them and, in short view, for the users. But on the long run, the User Experience declined.

The second example is from a computer game in the mid 2000s. It was a massive multiplayer online shooter with a decent implementation ahead of its time. But one thing was still from the last century: After each update of the game, the key bindings were reset to the default. And every other aspect of local modification to the settings, too. After each update, you had to configure your video, audio, controls and everything else like your in-game equipment again and again. The game didn’t offer any means to copy or reload your settings. It was up to you to maintain a recent backup of the settings files just in case. And if the file format was changed, you needed to combine their changes with yours. WinMerge is a decent tool for that task. But the problem is clear: The game developers couldn’t be bothered to think about how their upgrade strategy would affect their users. They ignored the problem and let the users figure it out. They chose better Developer Experience, free from complexities like user-side modifications over good User Experience like a game that can be upgraded without drawbacks for the gamers.

Sadly, this is an usual formula in software development: Developer Experience is more important than User Experience. Just look at it from an utilitarian point of view: If you burden thousands or even millions of users with just an easy one-minute task that you could fix during development, you have a budget of two workdays up to 10 person-years. Do the math yourself: One million users, each spending one minute of work on your software, equal to over 2000 workdays lost on a thing you could probably fix for everybody in an hour or two.

And this brings me to my central statement: Developer Experience, as opposed to User Experience, means nothing. It’s just not important. At least for all the users. It is important to the small group of developers and should not be forgotten. But never should a decision lead to better Developer Experience on the expense of User Experience. It’s a small inconvenience for us developers to think about smooth upgrades, meaningful and consistent control element titles or an easy installation. It’s a whole new game for our users.

Always choose User Experience over Developer Experience.

Call to action: If you have another good example of developers being lazy on the expense of their users, please share it as a comment.

Call to action 2: The initial picture is linked from the excellent website badhtml.com. If you ever feel like an imposter for your latest design, go and visit this website.