Category: Software Development

Documentation for your project: what and how

Writing documentation is seldom fun for developers and much useless documentation is written. I want to provide some guidelines helping to focus your project documentation efforts on useful stuff instead of following a set of dogmatic rules to plainly fulfill requirements.

Code Documentation

Probably written many times before but nevertheless often neglected:

  • Avoid untouched documentation templates, e.g. // This is a getter for A. They only clutter the code hurting developers instead of providing value.
  • Do not document every class, method, file etc. blindly. Focus on all API classes et al. to be used by other (external) developers.
  • Do not document what the code does, it should speak for itself. Rather explain why a certain algorithm or data structure is used. Try to communicate design decisions.
  • Check comments everytime you touch documented code and update them if necessary. Outdated documentation hurts more than its worth so if docs exists keep them up-to-date.

Project Documentation

This kind of documentation usually provides more value than many javadoc/doxygen generated pages. Nowadays, many people use a wiki software for project documentation. I encourage you to use a powerful wiki like Confluence because it provides rich formatting options and templating allowing for visually pleasing and expressive documentation. As such it may be even printed (to PDF) and handed out to your customers.

  • Putting parts like Installation into the code repository and integrating them into the wiki often serves administrators, managers (visibility!) and developers. See my older post “centralized project documentation” for some other ideas.
  • Wikis allow for easy editing and document sharing and are version controlled. All this facilitates reviews and updates of the documents.
  • Document prerequisites and external dependencies explicitly. They may be hard to find in configuration files but are of good use to people running your project.
  • Improve  searches in the wiki by providing tags and other metadata to help your future me and others finding the information they are looking for.
  • Provide consistent examples or even templates for common documentation tasks to encourage others and help them getting their project documentation started.

Conclusion

Good documentation is a real asset and can provide much value if you keep your efforts focused on the important stuff. Complex workflows and draconic rules will hinder documentation efforts wheres open collaboration and valuable documentation will motivate bringing more of it into existence.

When UTF8 != UTF8

Not all encoding problems are problems with different encodings

Problem

Recently I encountered a problem with umlauts in file names. I had to read names from a directory and find and update the appropriate entry in the database. So if I had a file named hund.pdf (Hund is German for dog) I had to find the corresponding record in the database and attach the file. Almost all files went smooth but the ones with umlauts failed all.

Certainly an encoding problem I thought. So I converted the string to UTF-8 before querying. Again the query returned an empty result set. So I read up on the various configuration options for JDBC, Oracle and Active Record (it is a JRuby on Rails based web app). I tried them all starting with nls_language and ending with temporary setting the locale. No luck.

Querying the database with a hard coded string containing umlauts worked. Both strings even printed on the console looked identically.

So last but not least I compared the string from the file name with a hard coded one: they weren’t equal. Looking at the bytes a strange character combination was revealed \204\136. What’s that? UTF8 calls this a combining diaeresis. What’s that? In UTF8 you can encode umlauts with their corresponding characters or use a combination of the character without an umlaut and the combining diaeresis. So ‘ä’ becomes ‘a\204\136’.

Solution

The solution is to normalize the string. In (J)Ruby you can achieve this in the following way:

string = string.mb_chars.normalize.to_s

And in Java this would be:

string = Normalizer.normalize(string, Normalizer.Form.NFKC)

Ruby uses NFKC (or kc for short) as a default and suggests this for databases and validations.

Lesson learned: So the next time you encounter encoding problems look twice it might be in the right encoding but with the wrong bytes.

The various ways of error handling

There are various approaches and philosophies regarding error handling in different programming languages. This article tries to give an overview.

Exceptions

Most of the current mainstream programming languages use exceptions for error handling. When an exception is raised (“thrown”), the call stack is being unwound until the exception is caught. The functions passed along the way through the call stack have to ensure that any opened resources are properly closed. This is usually done via finally blocks. If the exception is not caught the program terminates.

Exceptions come in two flavors: checked exceptions and unchecked exceptions. The handling of checked exceptions is enforced by the compiler. Checked exceptions are part of the function signatures. A function explicitly declares in its signature what exceptions can be thrown:

void f() throws A, B, C

The caller of a function has to either handle the exceptions (fully or partially) or let them pass through by re-declaring them in the throws clause of its own function signature.

Checked exceptions have the property that it’s hard to forget to handle them. However, proponents of unchecked exceptions argue that checked exceptions have two problems: versioning and scalability.

Once declared they are part of the interface and adding another exception will break all client code. Multiple exception types also tend to accumulate the more different subsystems are being aggregated. Proponents of unchecked exceptions prefer a catch-all clause further up the call stack. Some languages (e.g. Erlang) even follow a “let it crash” paradigm and simply respawn crashed processes. This approach is more viable in distributed systems than in user-facing applications.

Java is known for its checked exceptions. C#, C++, Scala and most dynamically typed languages decided to go with unchecked exceptions.

No exceptions

An alternative to exceptions is no exceptions. Exceptions overlay multiple different control flows, which makes it harder to reason about the control flow of a function. With exceptions functions can return at many other points than the explicit return points.

If an error is just a value that is returned by a function it can be handled by the usual control flow mechanisms of a language (like if and else) without the need of a special sub-language for error handling. These errors tend to be handled closer to the place of their occurrence rather than further up the call stack.

In such a language, which uses return values to flag errors, you’d better check all errors, otherwise you risk continuing with an incorrect, invalid or meaningless value. This can be enforced either by the compiler or via a lint tool.

There are different possibilities of how an error could be returned from a function:

In C a sentinel value in the range of the return type is often used to indicate an error, e.g. a negative value or zero. This is not a good solution, because it intermingles two things that do not belong together and it limits the range of valid return values. Another solution in C could be the use of an error output parameter. Prominent examples are NSError in Objective C or GError in GLib. This brings us to another possibility:

Some languages support multiple return values (e.g. Go) or tuple types (product types), which can act as multiple return values. One value can hold the actual result (e.g. number of bytes written), the other can indicate an error.

Multiple return types / product types are a simple solution, cover the necessary use cases and require little additional language support. A more sophisiticated and more restrictive solution are sum types, but they require a little bit more language support: instead of returning a value AND a possible error, a function returns either a value OR an error. This way the programmer is forced to check for an error by discriminating between the two cases. This is usually done via a feature called structural pattern matching (not to be confused with pattern matching on strings), either explicitly with a switch/case-like control structure or implicitly via convenience function. A popular example is Haskell’s Maybe monad or the Option or similarly named type in some other languages (e.g. Scala, Standard ML, OCaml).

 

Finally: return considered harmful

No wonder nobody wants to write explicit return statements anymore. They are more dangerous than you might think.

Ok, calm down already – I’m not going to take your return statement away. It’s a nice little statement that gets rarer and rarer,  mostly because many modern languages allow for implicit return values, e.g. in Scala, everything is an expression and the last expression of a method is its return value – except when it isn’t. You dont need to write an explicit return statement anymore, so our laziness takes over and we omit it.

And I’m not argueing that a finally block is a bad thing – quite the contrary. Even in the age of try-with-resources and autoclose closures, a well-crafted finally block is a beautiful piece of code at its right place.

But put the two together and you’ve got a recipe for desaster, at least in Java (try it in your language of choice with care). Let’s have a look at code:

public void doSomething() {
    try {
        throw new Error("drama!");
    } finally {
        // dispose resources here
    }
}

Ok, I could have spared you the drama of an Error and just thrown a RuntimeException (or a declared and checked Exception), but it’ll soon illustrate the effect all the better.

First, the code does exactly what it should: if you call the method, you’ll get to catch an Error and all resources that were used inside the method are cleaned up right before you catch it. And since the method has no return value, you don’t miss out there.

But what if I want to give one of those endangered return statements a new home and insert it into the finally block?

public void doSomething() {
    try {
        throw new Error("drama!");
    } finally {
        // dispose resources here
        return;
    }
}

Not much has changed. I’m still not missing any return value and the Error still gets thrown. The resources are cleaned up quite as thorough as before, but something important has changed: You won’t ever see the Error. You won’t catch it because it gets eaten by that return statement at the end of the finally block.

So in short: return statements are hungry little beasts that will eat your Throwables if you corral them inside a finally block.

No wonder nobody wants to have them in their code if they behave like that.

Oh, and rest assured that you will be warned: All modern IDEs will point you to the inevitable desaster waiting to happen, if only with a compiler warning.

But where do Throwables go when they got eaten? Good question. They don’t show up on any UncaughtExceptionHandler, they don’t even stay in memory. Probably, they just are digested, never to be seen again.

If you don’t regard return statements as a little more dangerous now, you probably also raise switch-statements for fun.

Grails Update from 2.2 to 2.3

An update in the minor version does not seem like a big step but this is one brought a lot of changes, so here a step by step guide which highlights some pitfalls.

An update in the minor version does not seem like a big step but this is one brought a lot of changes, so here a step by step guide which highlights some pitfalls.

First update the version of Grails in your application properties:

app.grails.version=2.3.8

The tomcat and hibernate plugins now have versions of their respective frameworks and not the version number of Grails:

plugins.tomcat=7.0.52.1
plugins.hibernate=3.6.10.13

Grails 2.3 has a new databinding mechanism. To use the old one, especially if you use custom property editors you have to add this option to your Config.groovy:

grails.databinding.useSpringBinder = true

But even with the old databinding something changed. The field id is not bound in command objects you need to bind id explicitly:

def action = { MyCommand command ->
  command.id = params['id']?.toLong()
}

Besides the databinding mechanism also the dependency resolving changed. But you can use the old ivy mechanism by including this in BuildConfig.groovy:

grails.project.dependency.resolver="ivy"

Nonetheless all dependencies must be declared in application.properties or BuildConfig.groovy. If you have a lib directory with local jars in your application you need to add this to your repositories as a local directory:

grails.project.dependency.resolution = {
    repositories {
        flatDir name:'myRepo', dirs:'lib'
    }
}

When you have all dependencies declared your application should start.

Tests

Grails 2.3 features a new test mode: forking. This causes some problems and is better to be deactivated in BuildConfig.groovy:

grails.project.fork = [
        test: false,
]

With the new version only JUnit4 style tests are supported. This means that you don’t extend GroovyTestCase or GrailsUnitTestCase. All rules must be public and non static. All tests methods need to be annotated with @Test. Set up methods are annotated with @Before and must be public. The tearDown methods must also be annotated with @After and be public. A bug in Grails prevents you from naming the set up and tear down methods freely: the names must be setUp and tearDown. All test methods must be public void, the old def declaration is not supported anymore. Now without extending GroovyTestCase you lose the assertion methods and need to add a static import:

import static groovy.util.GroovyTestCase.*

Unit Tests

All tests should be annotated with @TestMixin([GrailsUnitTestMixin]). If you need to mock domain classes you change mockDomain to @Mock:

class MyTest {
	public void testThis() {
      mockDomain(MyDomainClass, [mdc])
  }
}

@Mock([Proposal])
class MyTest {
	public void testThis() {
      mdc.save()
  }
}

Configuration is now already mocked and your properties can be added easily:

config.my.property.value.is='123'

Integration tests

As mentioned before setUp method naming has a bug: you have to name them setUp otherwise the changes to your database aren’t rollbacked.

Acceptance Tests with Selenium

You need to patch the Remote Control Plugin because of a ClassNotFoundException. Add an additional constructor to RemoteControl.groovy to support setting the classloader:

RemoteControl(ClassLoader loader) {
  super(new HttpTransport(getFunctionalTestReceiverAddress(), loader), loader)
}

In your tests you call this new constructor with the classloader of your class:

new RemoteControl(getClass().classLoader)

Using a Groovy Mixin in your application does not work in your tests and need to be replaced with grails.util.Mixin. But this only works in one way: the target class can access the mixin but the mixin not the target class. For this to work you need to let your mixin implement MixinTargetAware and declare a field named target:

class MyMixin implements MixinTargetAware {
	def target
}

Subtle changes and pitfalls

If you have a classname with a Controller suffix and a corresponding test but which isn’t a Grails controller Grails nevertheless tries to mock the class in your unit tests. If you rename the test to something without controller everything works fine.

In our pre 2.3 project we had some select tags in our views and used fieldValue for the selection:

<g:select value="${fieldValue(bean: object, field: 'value')}">

But now the select tag uses equals which fails if the values aren’t Strings. Just use the unescaped value:

<g:select value="${object?.value}">

I hope this guide and hints help others to avoid the headaches when upgrading your Grails application.

Using Rails with a legacy database schema – Part 2

Part one of this blog post mini-series showed how to override default table names and primary key names in ActiveRecord model classes, and how to define alias attributes for legacy column names.

This part will discuss some options for primary key definitions in the schema file, which are relevant for legacy schemas, as well as primary key value generation on Oracle databases.

Primary key schema definition

The database schema definition of a Rails application is usually provided in a file called schema.rb via a simple domain specific language.

The create_table method implicitly adds a primary key column with name id (of numeric type) by default.

create_table 'users' do |t|
  t.string 'name', limit: 20
  # ...
end

If the primary key has a different name you can easily specify it via the primary_key option:

create_table 'users', primary_key: 'user_key' do |t|
  t.string 'name', limit: 20
  # ...
end

But what if a table has a primary key of non-numeric type? The Rails schema DSL does not directly support this. But there’s a workaround: you can set the id option of create_table to false, declare the primary key column like an ordinary non-nullable column, and add the primary key constraint afterwards via execute.

create_table 'users', id: false do |t|
  t.string 'user_key', null: false
  t.string 'name', limit: 20
  # ...
end
execute 'ALTER TABLE user ADD PRIMARY KEY (user_key)'

Primary key value generation

On Oracle databases new primary key values are usually created via sequences. The Oracle adapter for ActiveRecord assumes sequence names in the form of table name + “_seq”.  You can override this default sequence name in a model class via the sequence_name property:

class User < ActiveRecord::Base
  self.sequence_name = 'user_sequence'
  # ...
end

Sometimes primary key values are auto-generated via triggers. In this case you need the Oracle Enhanced adapter, which is a superset of the original ActiveRecord Oracle adapter, but with additional support for working with legacy databases. Now you can set the sequence_name property to the value :autogenerated:

class User < ActiveRecord::Base
  self.sequence_name = :autogenerated
  # ...
end

This circumvents the default convention and tells the adapter to not include primary key values in generated INSERT statements.

A coder’s manifesto

A personal manifesto about what I value in developing software and what I think needs to change how we develop software.

Remember the Agile manifesto? What was the most important principle behind it?

User value first

Our highest priority is to satisfy the customer
through early and continuous delivery
of valuable software. – The agile manifesto

If it doesn’t benefit the user it should not be done. This can be a new feature, a better user interface, a clearer wording, better performance, robustness, … Not all improvements have immediate value but they must have a value at some time. If you look at the craftsman project priorities timely delivery has the most user value for me. (Personal footnote: I think this is where the software craftsmanship movement sets the wrong focus: quality is not the most important thing, user value is). But how do you know what the user might need?

Communication is key

Communication in all parts of software development is a must. You need to talk to your users, your fellow developers and other project partners. What often is neglected is the communication part of the code and the documentation. The primary measure of good code is how well it communicates its intent. How clear it is. Many measure code quality with things like testability, low coupling, coverage, metrics. But clarity and communication are by far the most important. If you can understand what the code does and how and why, you are able to change it. I personally believe that not only the statements but also the formatting of the code and the individual expression, the style, is important. Just as in novels and poetry, the typography emphasizes the meaning, the formatting can do this for the code.
Sometimes the why of decisions cannot be expressed in code this is where documentation comes in. If you have a need to document the what or the how, refactor your code. The importance of communication cannot be understated. Often talking with the user first can save you days and weeks of coding. How?

KISS and YAGNI

Simple and easy. As computer scientist we are trained to solve a problem correctly in all its glory. Every corner case is handled and secured by a test. I think to be more successful as a developer we have to unlearn this. (don’t get me wrong: there are places and software where we need this but the majority of software doesn’t need this) How often did I implement a complete version of a solution to later see that only 80% of it was used. The remaining 20% occur once in a lifetime but needed the most of the development time. If we constraint the problem we can save magnitudes of development time and can invest in more user value. And in…

Developer happiness

Some developers trade development productivity for runtime performance. But we don’t want to code in assembler or C. I happily trade runtime performance for productivity. Productivity means happiness. I am happy when I get things done. If I need to improve the performance I can focus on the parts which need it (remember YAGNI?). Programming languages, frameworks and platforms are not just tools, they are and form our ways of thinking. Tools we use have to help us reaching our goals, so…

Testing is important but just a tool

Testing gives me confidence to change code without breaking things. It helps me to avoid regression. Tests are a great tool. But only a means to an end. The program code is more important than tests, so tests should not force me to make compromises in clarity or communication level of the code. Ideally testing environments should be easy to set up and execute. One thing the recent TDD debate showed me is that we need to focus on the goals we have and the problems we want to solve with our tools. The tools we use should take a front row seat. If they need more focus or effort than the benefit they bring something is wrong. We need to constantly assess if the tools help us to reach our goals. So we have to…

Reflect

The last principle of the agile manifesto is to reflect regularly. What can be done better? How to improve? What did we learn? Often this sounds like a chore. Many times the reflection is omitted. Clean code tries to prevent this with daily reflection. But the set of practices and principles is questionable and carved into stone. So what can be done to make reflection more attractive? One of my suggestions is to keep a

Developer handbook

Reading Small Talk Best Practice Patterns (by Kent Beck) I could not help but have a feeling that this is like a personal developer handbook. In it Kent touches many important aspects of programming like composition of methods, naming and formatting. On top of that he describes his personal experiences and opinions in many of the patterns. I found this really valuable. I think starting with some of these patterns and my own experiences it would be helpful to record them in a personal developer handbook. I can use this book to remember past solutions and reflect on them. I can add my experiences in different projects and contexts to these. The goal is to get better at writing clear code and improve the communication of my code. These records could also help me to make my common habits, patterns, mistakes and approaches to problems explicit and learn from them. This is a personal book but it could also be a team effort or help others. The last part, the conclusion, of the TDD debate has a special insight for me: I should not lean on the masters of software development to advance our field and my development skills in particular. I need to find my own way and many things I take for granted I have to…

Re-think

Kent told an anecdote how he discovered TDD and then it struck me: he had a goal (or a need) in mind and had to find his way. In hindsight it sounds easy but it takes courage and persistence to push through. I think many more problems currently exist in developing software and often we took them for a given, we adjusted us, we accepted them. We cannot imagine a solution for these problems because like the elephant with the rope we never experienced a time without them. But this needs to change. We have to rethink the unsolved or inadequately solved problems and ignore some conventions and habits we formed as developers and as an industry. We have to rethink some of our approaches and assumptions. I believe we can find new ways which improve how we develop software and for this we need to rethink.

Don’t let one bad apple spoil the whole bunch

One exception in a collection operation like for-each or map/collect stops the processing of all the other elements. Instead of letting the whole task blow up it is often more desirable to skip those elements causing failures, log the errors (and possibly notify the user about the failing elements), but have all other elements processed. Examples for such operations are: sending bulk mails to users, bulk import/export, lists in user interfaces etc., and common errors are, for example, NullPointerExceptions, database errors or wrong email addresses.

Here’s some simple code for robust and reusable for-each and map operations in JavaScript:

function robustForEach(array, callback) {
  var failures = [];
  array.forEach(function(elem, i) {
    try {
      callback(elem, i);
    } catch (e) {
      failures.push({element: elem, index: i, error: e});
    }
  });
  return failures;
}

function robustMap(array, callback) {
  var result = { array: [] };
  result.failures = robustForEach(array, function(elem, i) {
    result.array.push(callback(elem, i));
  });
  return result;
}

Similar code can be easily implemented in other languages like Java (especially with Java 8 streams), Groovy, Ruby, etc.

If you decide to log the errors, you have to choose between two possible log strategies: one log operation per error, which can be annoying if you get a mail for each logged error, or one log operation bundling all occurred errors (make sure that a failing toString can’t spoil the whole bunch again).

function logAny(failures) {
  failures.forEach(function(fail) {
    log.error(failMessage(fail));
  });
}

function logAnyBundled(failures) {
  if (failures.length == 0) {
    return;
  }
  log.error(failures.map(function(fail) {
    return failMessage(fail);
  }).join('\n'));
}

function failMessage(fail) {
  return "Could not process '" +
         fail.element + "': " + fail.error;
}

You can easily combine the map and log operations:

function robustMapAndLog(array, callback) {
  var result = robustMap(array, callback);
  logAny(result.failures);
  return result.array;
}

Example usage:

var numbers = [1, 2, 3, 4, 5, 6, 7, 8];
var result = robustMapAndLog(numbers, function(n) {
  if (n == 5) {
    throw 'bad apple';
  }
  return n * n;
});
print(result);

// Error log output:
//  Could not process '5': bad apple
// Output:
//  [ 1, 4, 9, 16, 36, 49, 64 ]

One element could not be processed due to an error, but all other elements were not affected.

Conclusion

Be aware of the bad apple possibility for every loop you write (explicitly or implicitly) and consciously choose the appropriate error handling strategy depending on the situation. Don’t let indifference decide the fate of your bulk operations.

How the most interesting IT debate is revealing our values as software developers

TDD is dead. Is TDD dead? A question that seems to divide our profession. What does this debate have to do with you?

TDD is dead. Is TDD dead? A question that seems to divide our profession.
On the one side: developers which write their tests first and let them drive their code. They prefer the mockist approach to testing. Code should be tested in isolation, under lab like circumstances. Clean code is their book. Practices and principles guide their thinking. An application should not be bound to frameworks and have a hexagonal architecture. The GOOS book showed how it can be done.
On the other side: developers which focus on readability and clarity. They use their experience and gut to drive their decisions. Because of past experiences they test their the code the classical way. They are pragmatic. Practices and principles are used when they improve the understanding of the code. Code is there to be refactored. Just like a gardener trims bushes and a writer edits his prose they work with their code.

What are your values?

What does this debate have to do with you?

Ask yourself:
What if you could write a proof of your program costing 10 or just 5 times as much as the implementation? It would prove your code would work correctly under all possible circumstances. Would you do it?

Or would you rather improve the existing architecture, design or clarity of your code? So that you remove technical debt and are better positioned for future changes.

Or would you write new features and improve your application for the people using it?

What are your values?

History

At the beginnings of my developer life in the late 80s/early 90s I remember that the industry was focussed on one goal: code reuse. Modules, components, libraries, frameworks were introduced. Then patterns came. All of that was working towards one side of the equation: low coupling.
High cohesion was neglected in pursuit of a noble goal. But what happened? The imbalance produced layer after layer, indirection after indirection, over-separation and over-abstraction. You had to deal with dependency injection (containers), configuration, class hierarchies, interfaces, event buses, callbacks, … just to understand a hello world.
Today we have more computing power and are solving more and more complex things. We think in higher abstractions. Much more people benefit from our skills and our works.
On the user facing side design focusses on simplicity and usability. Even complex relationships can be made understandable and manageable. A wise man once said: design is about intent.
The same with code: Code is about intent. Intent should be the measure of the quality of our code. Not testability, not coupling: intent. If the code (and this includes the code comments) would reveal its intent, you could fix bugs in it, improve it, change it, refactor it. Tests would be your safety net to ensure you are not breaking your intent.
You might say: but this is what TDD is all about! But I think we got it all backwards. The code and its intention revealing nature is more important than the tests. The tests support. But tests should never replace or even harm the clarity of the code.
The quality of the code is important. But most important are the people using your application.
My goal is to delight the people who use my software and my way there is writing intention revealing software. I am not there and I am learning every day but I take step after step.

What are your values?

A hierarchy of project needs

We all know Maslow’s pyramid, so why not apply the idea to the needs of a software development project (note: not the developer of the project!).

A few weeks ago, I traded stories with a fellow software developer when he told me this little gem: A developer programs a web shop that looks pretty and runs smooth. But as soon as you place multiple items in the shopping cart, you’ll inevitably end up with an amount of XX.999999999998 euros (or whatever currency you want). When asked why the shopping cart “computes the wrong amounts”, the developer answered that the amount is correct and that’s just the way a floating point number behaves. He didn’t see a problem with the functionality. My immediate answer: “Wow, that’s very low on the Maslow pyramid”. We both understood, but since then, I tried to come up with a Maslow-like pyramid that would explain my sentence to a larger crowd. So here is how my attempt has grown so far.

Maslow’s hierarchy of needs640px-Maslow's_Hierarchy_of_Needs.svg

Abraham Maslow was an american psychologist that studied mental health and human potential. He invented an hierarchy of human needs that is also known as Maslow’s pyramid. On a side note, he also pointed out the human tendency to over-apply known tools. His pyramid has five stages (IT people would call them layers) of human needs that begin with the very basic ones (e.g. air, water) and scales to the abstracts like morality and creativity. If something is “low on the pyramid”, then it can be seen as granted by priviledged people. Most of us never think about our air supply requirements. Everything “high on the pyramid” can be seen as “expendable” in times of crisis. Morality will be forgotten as soon as we seriously lack water.

A hierarchy of project needs

My immediate answer to the story in the introduction suggested that I think an equivalent pyramid exists for the needs of a software development project. And a quick research on the internet reveals that I’m not the only one with that idea. For example, Scott Hanselman blogged about it in 2012, and Francis Shanahan came up with an extended version in 2009. Both adaptions are reasonable and stand on their own – I don’t want to invalidate or change them. Instead, I publish my attempt as an addition to the discussion, if there is any.

Here is my five-layered pyramid of project needs:

project maslovLayer 0: Executable

Let’s face it: If your project doesn’t compile or crashes right after being started, it isn’t much worth. And just because it runs on your machine doesn’t make it any more useful to others. So the most basic need a project has is to be executable on the target machine. This includes some form of correctness – if your program doesn’t perform the right operations, it can run indefinitely and not provide any value. Please note that the program doesn’t have to be bug-free or tested to be useful. It just has to adhere to the intended use case. In our introduction story, the web shop looks pretty and runs smoothly. It certainly is “Executable”.

Layer 1: Abstraction

This is where I placed the mishap in the introduction story. Every project needs some form of abstraction or separation between the internal representation of data and functionality and the external presentation to the user. This is probably trivial to most of you, but I’ve seen way to much code that uses external presentations (e.g. strings from the GUI) to make important decisions and others have, too. A key rule is “once data is formatted, it is eternally lost and unavailable to computing / data processing“. The rule for the other way is that you should never present data without proper (human-readable) formatting. The amount of work you save by not pretty-printing (formatting is just the formal term for adding syntactic sugar to make the data edible for humans) is largely offset by the amount of work your users will have to invest to decipher the output.

Layer 2: Architecture

You can call it design, architecture or whatever you like, any reasonably large code base needs some kind of structuring that prevents it from imploding. A whole theory of patterns was invented to keep code aerated enough to prevent it from decomposing to compost. And we all know what compost code looks and smells like. Applying architecture to your code keeps it maintainable and refactorable and in outstanding cases even modularizable. This is the layer where most projects fail on the long run. Even if at first there was a design, it gets watered down with every modification. Good principles to counter this effect are the “no broken windows” approach and the boy scout rule.

Layer 3: Verification

There is a moment in programming when you hand your code over to the next developer. Usually, this moment is called “commit” (if you don’t use version control, have a good look at Scott Hanselman’s lowest pyramid layer!). Oftentimes, the next developer is future you – and you have no clue what past you thought when he wrote that crap. You can’t even distinguish between features and bugs. That’s why your project wants verification. It’s not utmost important if you verify your code with unit tests, integration tests, acceptance tests, contracts or all of them together. It’s important that your code is accompanied by automated guardian angels that catch the most dangerous accidental modifications and help to point out the bugs among the features. Automated verification tells future you that whatever past you wanted to build, it’s still intact. This layer is the life insurance for functionality as much as the architecture layer was for code.

Layer 4: Style

Every program in the world can still do its job properly even if we would eliminate everything “stylish” in their codebase. Style is the most human-centered need in the pyramid. No machine or compiler has yet developed aesthetic likings. Scott Hanselman called this layer “bragging rights”, another thing computers don’t care about. This is the level where most bickering among developers takes place, but it’s also the level that can most easily be ignored without sacrificing critical project needs. Or, to put it bluntly: Your project most likely doesn’t care half as much about style as you do.

Where to go from here?

My most important message with the hierarchy of project needs is that we often focus on the higher needs and take the lower ones too much for granted. If your code lacks in the fundamental layers, the damage is much greater in terms of project value. A stylistic displeasing code will hurt the next developer, but a code lacking abstraction will hurt every user of your software, as exemplified by the story in the introduction. As we developers should be the advocates of our project’s needs, we have to think more in regard of its benefit than our personal self-actualization. But the required traits to do so properly aren’t even on the original Maslow’s pyramid, so it’s a big challenge for any of us.