Zero Interaction Tools

Some time ago, a customer called us for a delicate task: To develop a little tool in a very tight budget that aggregates pictures in a specific way. The pictures were from the medical domain and comprised of sets with thousands of pictures that needed to be combined into one large picture with the help of some mathematics. Developing the algorithm was not a problem, the huge data sizes neither, but the budget was a challenge. We could program everything and test it on some sample picture sets (that arrived on several blue-ray discs) within the budget, but an elaborate graphical user interface (GUI) would be out of scope. On the other hand, the anticipated users of the tool weren’t computer affine enough to handle a CLI (command line interface). The tool needed to be simple, fast and cheap. And it only needed to do one thing.

Traditional usage of software tools

In the traditional way, a software tool comes with an installer that entrenches the tool onto the target computer and provides a start menu entry, a desktop icon and perhaps even a boot launcher with a fancy tray icon and some balloon tooltips that inform the user from time to time that the tool is still installed and wants some attention. If you click on the tool’s icon, a graphical user interface appears, perhaps in the form of an application window or just a tray pop-up menu. You need to interact with the user interface, then. Let’s say you want to combine several thousand pictures into one, then you need to specify directories or collections of files through some file dialogs with “browse” buttons and complex “ingredients” lists. You’ve probably seen this type of user interface while burning a blue-ray disc or uploading files into the cloud. After you’ve selected all your input pictures, you have to say where to write to (another file dialog) and what name your target file should have. After that, you’ll stare at a progress bar and wait for it to reach the right hand side of the widget. And then, the tool will beep and proudly present a little message box that informs you that everything has worked out just fine and you can find your result right were you wanted to. Afterwards, the tool will sit there on your screen in anticipation of your next move. What will you do? Do it all again because you love the progress bars and beeps? Combine another several thousand pictures? Shutdown the tool? Oh, come on! Are you sure you want to quit now?

None of this could be developed in the budget our customer gave us. The tool didn’t need the self-marketing aspects like a tray icon or launcher, because the customer would only use it internally. But even the rest of the user interface was too much work: the future users would not get a traditional software tool.

Zero interaction tool

So we thought about the minimal user interface that the picture aggregation tool needed to have. And came to the conclusion that no user interface was needed, because it really only needed to do one thing. At least, if certain assumptions hold true:

  • The tool is fast enough to produce no significant delay
  • The input directory holds all pictures that should be aggregated
  • The input directory can be the output directory as well
  • The name of the resulting picture file can contain a timestamp to distinguish between several tool runs

We consulted our customer and checked that the latter three assumptions were valid. So, given we can make the first assumption a reality, the tool could work without any form of user interaction by being copied into the picture directory and then started.

Yes, you’ve read this right. The tool would not be installed, but copied for every usage. This is a highly unusual usage scenario for a program, because it means that every picture directory that should be aggregated holds an identical copy of the program. But if we can make some more assumptions valid, it is a viable way to empower the users:

  • The tool must run on all target machines without additional preparation
  • The tool must only consist of one executable file, no DLLs or configuration files
  • The tool must be small in size, like one megabyte at most

We confirmed with a quick and dirty spike (an embarrasingly inchoate prototype) that we can produce a program that conforms to all three new assumptions/requirements. The only remaining problem was the very first assumption: No harddrive was fast enough to provide the pixel data of thousands of pictures in less than a second. Even if we could aggregate the pixels fast enough (given enough cores, this would be possible), we couldn’t get hold of them fast enough. We needed some kind of progress bar.

Use your information channels

We thought about the information channels our tool would have towards the user. Let’s repeat the scenario: The user navigates to the directory containing the pictures that should be aggregated, copies the executable program into it and double-clicks to start the tool. There are many possibilities to inform the user about progress:

  • Audio (Sound): We can play a little tune or some sound that changes frequency to indicate progress. This is highly unusual and we can’t be sure that the speakers aren’t muted (usage on a notebook was part of the domain analysis results). No sounds, that is.
  • Animation (Graphics): In the most boring case, this would be a little window with a progress bar that runs from left to right and disappears when the work is done. Possible, but boring. Perhaps we can think of something more in tune with the rest of the usage scenario.
  • Text: Well, this was the idea. We produce a result file and give it a name. We don’t need to keep the name static as long as we are working and things change inside the file, anyways. We just update the file name to reflect our progress.

So our tool creates a new result file in the picture directory that is named result_0_percent or something and runs up to result_100_percent and then gets renamed to result_timestamp with the current timestamp. You can just watch your file explorer to keep up with the tool’s completion. This is a bit unusual at first, but all pilot users grasped the concept immediately and were pleased with it.

The result

And this is the story when we developed a highly specialized tool within a very small budget without any graphical or otherwise traditional user interface. The user brings the tool to the data (by copying it into the same directory) and lets it perform its work by simply starting it. The tool reports its progress back via the result file name. As soon as the result file contains a timestamp (and the notebook air fans cease to go beserk), the user can copy it into the next tool in the tool chain, probably a picture viewer or a printer driver. The users loved the tool for its speed and simplicity.

One funny side-note remains to be told: Because thousands of pictures aggregated into one produces a picture with a lot of details, the result file was not too big (about 20-30 megabytes), but could take out any printer for several hours if printed. The tool got informally renamed to “printer-reaper.exe”.

Makeup on a zombie – Java Swing UX improvements

When I learned Java programming in 1997, the AWT classes were the default way to create graphical user interfaces. The AWT widgets were not very sophisticated and really ugly, so it is no surprise they were replaced by a new widget toolkit, called “Swing”, as soon as possible. At the end of 1998, the Swing graphical API was the default way to develop GUIs for desktop applications on the Java platform.

Today, twenty years later, the Swing API is still part of the Java core SDK and ready for your adventures in GUI creation. But time has taken a toll on the technology. The widgets, once displayed with a state-of-the-art design, look really outdated. Swing introduced the concept of pluggable “Look-and-Feels” (L&F), so you could essentially re-skin your interface with a few lines of code, but all L&Fs look ugly and feel cumbersome now. You can say that Java Swing is a zombie: It is still available and in use in its latest development state, but makes no progress in regard of improvements. If software development follows one rule, it is that software that isn’t actively developed anymore is dead.

My personal date when Java Swing died was the day Chet Haase (author of the Java Swing book “Filthy Rich Clients”) left Sun Microsystems to work for Adobe. That was in 2008. The technology received several important updates since then, but soon after, JavaFX got on the stage (and left it, and went back on, left it again, and is now an optional download for the Java SDK). Desktop GUIs are even more dead than Java Swing, because “mobile first” and “web second” don’t leave much room for “desktop third”. Consequentially, Java FX will not receive support from Oracle after 2022.

But there are still plenty of desktop applications and they won’t go away anytime soon. There is a valid use case for a locally installed program with a graphical user interface on a physical computer. And there are still lots of “legacy systems” that need maintenance and improvements. Most of them are entangled with their UI toolkit of choice – a choice made before 2007, when “mobile first” wasn’t even available as an option.

Because those legacy systems still exist and are used, their users want to experience the look and feel of today’s applications. And this is where the fun begins: You apply makeup on a zombie to let it appear a little bit less ugly than it really is.

Recently, my task was to improve the keyboard handling of a Java Swing desktop application. It was surprisingly easy to add a tad of modern “feel”, and this gives me hope that the zombie might stay semi-alive longer than I thought. As you might already have guessed, StackOverflow is a goldmine for answers on ancient technology. Here are my first few improvements and their respective answer on StackOverflow:

  • Let’s suppose you want or need to interact with your application without a mouse or touchscreen. Your first attempt to start an interaction is to press the “menu” key in order to activate the application menu. This would be the “Alt” key on a windows system. For modern applications, your input focus is now at the menu bar. In Java Swing applications, nothing happens. You have to press “Alt” and a mnemonic character to enter a specific menu. If you want to reduce the initial hurdle to just one key, you need to teach all your Java Swing menus to react to the “Alt” key alone: https://stackoverflow.com/a/8659116
  • Speaking of focus, in modern applications you can move your focus by using the arrow keys. Java Swing still thinks that “Tab” and “Shift+Tab” is the pinnacle of focus control. If you want to improve the behavior (and therefore the “feel”) of your focus traversal, you can do it globally for your application: https://stackoverflow.com/a/8255423
  • And if you want to enable the Return/Enter key for button activation, you can do it with just one line: https://stackoverflow.com/a/440536

If you happen to work on a Java Swing application and want some cheap user experience upgrades, I’ve assembled all the knowledge above into a neat little class that you can use as an add-on utility class: https://github.com/dlindner/java-swing-ux/blob/master/src/com/schneide/swing/ux/KeyboardUX.java

What are your makeup tips for zombies?

UX is a mindset, not an engineering task

With all those methods and measurements like A/B testing, eye tracking and so on you would believe you can engineer your way to a perfect UX but that isn’t what matters. The user and his experience matters in the end and this is delivered by the product which in turn reflects your mindset. Just like the Conway’s law which states that the architecture of your software reflects your architecture of your organization, the product’s design and user interface reflects your mindset.
But what mindset is this? Let’s take a look at my UX posts of the past.

For your last project ask yourself what did the stakeholders learn

There two lessons for UX in here: UX design is a collaborative effort and learning is really important.

How do I start a project
and
How I start a project – the next steps

UX is an iterative way to explore a problem space. It has a goal: meeting the users’ needs. And again: a collaborative one: we need a shared understanding between all the project’s participants.

Quick and dirty is a skill

Evaluating is key in UX, and for not overwhelming the effort to do so, we need to find quick and sometimes dirty solutions to test our hypotheses.

The definition of done

Meeting the spec isn’t a goal of UX, meeting the user’s needs and goals is.

Personas – the great misunderstanding

Tool is just a tool is just a tool. It can help to frame your thinking but it cannot replace your thinking.

Mapping the users workflow

Another tool which can help to connect the disconnected parts, the user stories or issues, to a whole. IN this way you see your software from the user’s perspective from his way through your interface.

What UX and sales have in common

The user is central and context is key.

Discount UX

Again: UX does not need fancy tools, the mindset is really important and you should use the tools you have: pen, paper and your brain.

From agile to UX – a change in perspective

Focus on the user and his tasks, try to formulate the requirements from the user’s perspective.

Requirements should not drive development

Jobs should. Jobs are tasks the users wants to do in a specific context. These define what the software should do when it is ready.

UX is like a text adventure

You start with a beginner’s mind, try not to assume anything.

Learning UX: where do I start

Start with listening with an open mind and think.

Assumptions how to find, track and eliminate them

Beware of your bias.

Transparent software: making complexity understandable

Complexity isn’t your enemy. Find the essential complexity that you need to reach your user’s goals.

What developers can learn from designers

Slow down, do not rush towards your goal. Software is intent. Build to learn. Focus on the whole more than the parts. Have and provide alternatives.

Developer Experience means nothing

You’ve probably already heard about the concept of User Experience (UX) and the matching virtue of User Experience Design. If not, you might want to go check it out. I would suggest you start with the excellent book “The Design of Everyday Things” from Donald Norman. It has nothing to do with software development, so you won’t mix up User Experience Design with Graphical User Interface Design or even Graphics Design.

In short, User Experience is what an user of your software will experience while using your software. If your software makes them smile, you’re some kind of UX god. If your software makes them curse repeatedly, you’re probably not. You can try to improve the experiences of your users by making changes to the software. This is called User Experience Design and is an important part of software development. Most developers know nothing about User Experience Design.

What developers naturally know about is Developer Experience (DX), a concept that isn’t really defined in the literature. I made it up to explain my point to you. Every software developer has a favorite programming language and IDE. Remember, the source codes of the same problem in different programming languages will ultimately yield the same machine code. The machine is totally agnostic about your preferences for a certain programming language. Your choice of a programming language for a certain problem is more about your Developer Experience than anything else. Developer Experience is everything you feel and think during software development. A bad Developer Experience lets you swear about the tools, the code and everything and everybody else around you. A good Developer Experience gives you a sense of accomplishment and safety while coding and makes your work not harder than necessary.

Because you can’t read elsewhere about the concept of Developer Experience, I want to give two more examples and show how it affects the User Experience. The first example is a big, long-lasting software system that is in production and still developed further. If you have a software system in production, everything requires an additional thought about the matching upgrade strategy. You can’t just modify the database structure, you need to provide migrations. You can’t just add a configuration entry, you need to make it optional or consider the least harmful default value. In the project of our example, the developers had three possible approaches to a new configuration entry:

  • They could add it to the code but leave the configuration files unchanged. This required the code to handle the “absent from configuration” case in a useful manner. It required the developer to make effort in the code. The user would not know about the new configuration entry if not stated in some external documentation.
  • They could add it to the code and write a configuration migration script that added it to the existing configuration files on the customer’s installation. The code could now expect the entry to be present, but the developer had to write and test the migration script code. The user would see the new configuration entry with the default value.
  • They could introduce a new configuration file to the system and place the new configuration entry in it. The code could expect the entry to be present, because new configuration files were added to an existing installation during the upgrade process. The user would see the new configuration file and the new configuration entry with the default value.

You can probably guess which of the three options got used so excessively that users complained about the configuration being all over the place, in a myriad of little one-liner configuration files with ominous names. The developers chose the best option for them and, in short view, for the users. But on the long run, the User Experience declined.

The second example is from a computer game in the mid 2000s. It was a massive multiplayer online shooter with a decent implementation ahead of its time. But one thing was still from the last century: After each update of the game, the key bindings were reset to the default. And every other aspect of local modification to the settings, too. After each update, you had to configure your video, audio, controls and everything else like your in-game equipment again and again. The game didn’t offer any means to copy or reload your settings. It was up to you to maintain a recent backup of the settings files just in case. And if the file format was changed, you needed to combine their changes with yours. WinMerge is a decent tool for that task. But the problem is clear: The game developers couldn’t be bothered to think about how their upgrade strategy would affect their users. They ignored the problem and let the users figure it out. They chose better Developer Experience, free from complexities like user-side modifications over good User Experience like a game that can be upgraded without drawbacks for the gamers.

Sadly, this is an usual formula in software development: Developer Experience is more important than User Experience. Just look at it from an utilitarian point of view: If you burden thousands or even millions of users with just an easy one-minute task that you could fix during development, you have a budget of two workdays up to 10 person-years. Do the math yourself: One million users, each spending one minute of work on your software, equal to over 2000 workdays lost on a thing you could probably fix for everybody in an hour or two.

And this brings me to my central statement: Developer Experience, as opposed to User Experience, means nothing. It’s just not important. At least for all the users. It is important to the small group of developers and should not be forgotten. But never should a decision lead to better Developer Experience on the expense of User Experience. It’s a small inconvenience for us developers to think about smooth upgrades, meaningful and consistent control element titles or an easy installation. It’s a whole new game for our users.

Always choose User Experience over Developer Experience.

Call to action: If you have another good example of developers being lazy on the expense of their users, please share it as a comment.

Call to action 2: The initial picture is linked from the excellent website badhtml.com. If you ever feel like an imposter for your latest design, go and visit this website.

UX tips: charts

Continuing our series about UX tips for small areas we take a look at charts this time.

  • always label your axis – this one should be obvious the reader needs to know at what data he looks
  • use units – is this meter or yards, data can only be understood if the scale is clear
  • use distinctive colors – different data, different color, and don’t forget the color blind
  • use color with purpose – same data, same color, e.g. having different values of the same source should use the same color or tint of color
  • for bar charts always start at zero – the distortion often used for exaggerating differences is immense
  • leave out decoration – decoration distracts from the data
  • no 3d effect – data speaks for itself, please do not try to make it prettier
  • use them for lots of data – if you just have a few numbers a table of data might be more appropriate
  • use a title – what aspect or slice of data are you showing me, a summary helps to understand perspective and intention
  • use a big enough font – this and the appropriate contrast should be obvious for all UI work
  • label tick marks – use them at meaningful intervals, do not just slice the axis into equidistant intervals
  • include time – even if it isn’t a time series chart, tell the reader from when the data is
  • size 2d shapes by area – do not use diameter or radius for sizing 2d shapes (circles, bubbles…), unless you use bar charts (where the length is compared) the proportion gets way out of hand
  • use consistent rounding – first round your numbers and do this consistently
  • for comparisons use the same scale – do not distort your data by presenting it at different scale

UX tips: Forms

User experience is a vast field which can be overwhelming at start. To make it easier for others I want to break it down to specific areas.
The start makes a rather narrow field of software: forms.

Forms are ubiquitous: almost every software user interface has them. Most of them are too big and overwhelm the user. But in complex software you cannot “just” leave out some inputs to make a small form. Here are some tips to improve your next form:

  • use a grid – your labels and inputs (and indeed every UI element) should be layouted on a grid, the goal is to improve scanability and readability and to reduce visual clutter
  • align all labels in the same way – this one should be obvious, but often it is missed, it doesn’t matter if the labels are left or above the input, all should be aligned in the same way
  • use labels – another obvious one but often labels are omitted to make the UI look cleaner, but if the user cannot see from looking at your interface where he inputs his username or password something went really wrong
  • put fields in chunks – if the form gets too big (and most of them do), use blocks with whitespace around them to chunk fields, how do find out the groups for the chunking? You should know the domain and you can always ask the user
  • use specialized inputs – if only a date can be entered use a calendar widget, if you need a color use a palette input, the goal is to reduce errors made by the user, which also reduces his frustrations
  • provide format helps – if you cannot provide a specialized input, provide format helps, describe how the input should be formatted and what formats are accepted, again to reduce errors
  • order the fields – ask the user and a domain expert what the mental model of the fields is, what order should the inputs be made, what is optional, what is important
  • distinct the mandatory from the optional – nothing is more frustrating than to fill out a form, click submit and get told 10 times which fields are missing
  • use different sizes – if the input is just a one digit number the input should be sized to indicate this, if you want 3 lines of text, use three rows, the goal is to visually communicate what kind of input is expected, but remember: please align them properly

I hope these tips help you to make better forms and make your users less frustrated.

For your last project ask yourself: What did the stakeholders learn

At the start of a new project we like to begin with a naive mind, a beginner’s mind. In it we try to avoid our assumptions and start with a blank slate. Our clients do not. They are expert in their respective domain and know a lot. It’s naturally that during the project we learn lot about them and their domain, their work and their daily struggles. We see how they work around the limitations of their tools and cope with software written more than 30 years ago.
But besides us learning something about the domain, the stakeholders learn something about their domain, too. Because to develop the domain, the use cases and the daily work, we have to know details and reasons. Why is this step before that? Is it optional? Are these all the formats which are allowed? How long is the text usually? Why is there an exception to the rule? How often does it happen?
Usually we ask questions which cover the most traveled path, the happy trail. But in order to understand we need to get to the edges as well. The dark edges. Sometimes the number of objects we deal with is so big, nobody has all the answers. Our work, even before we write the software, enables collaboration. People and different departments have to work together. We work with all of them. Our software helps them to reach their common goals. But before that we need to know. And in order to tell us that the stakeholders need to dig deeper in their respective domain. Sometimes we need to look at the history in their domain, their work history, the decisions other stakeholders made in the past. It’s like archeology without the shovels, well, most of the time :).
Luckily the people we work with enjoy getting to know more about their work. They are astonished what depth the details have. How much different types of things, where gaps are. It is not always easy to light up areas that were kept in the dark so long. That were done just the way they were done. No we come and ask sometimes uneasy questions. We need to know. We need to know exactly. We need to know deeply.
This curiosity is not for its own sake. Our clients can confirm that the new software is so much better than the old. Not technical, but most importantly more adapted to their daily work.

That’s what’s important.