How to approach big tasks

In the heart of software development lies “the system”. The system is always complicated enough that you cannot fully grasp it and it is built by stacking parts on top of another that are just a tad too big to be called simple. The life of a software developer is an ongoing series of isolated projects that are at the threshold of his or her capabilities. We call these projects “epics”, “stories” or just “issues”. The sum of these projects is a system.

Don’t get me wrong – there a tons of issues that just require an hour, a cup of coffee and a few lines of code. This is the green zone of software development. You cannot possibly fail these issues. If you require twice the time, it’s still way before lunchtime. And even if you fail them, a colleague will have your back.
I’m talking about those issues that appear on your to-do list and behave like roadblocks. You dread them from far away and you know that this isn’t smooth sailing for an hour, this will be tough work for several days. This isn’t just an issue, it is an issue by itself for you. You are definitely unsure if you can make it.

Typical small project management

How do you approach such a project? It isn’t an issue anymore, as soon as you get emotionally involved, it becomes a project. Even if your emotion is just dread or fear, it is still involvement. Even if your management style is evasion, it is still project management. Sure, you can reassign a few of these icebergs, but they will always be there. You need to learn to navigate and to tackle them. Hitting an iceberg in the “frontal collision”-style isn’t a good idea.

On closer inspection, every project consists of numerous parts that you already know a solution for – typical one-hour issues – and just a few parts that you cannot estimate because you don’t know how to even start. Many developers in this situation take the route of least resistance and start with the known pieces. It’s obvious, it feels good (you are making good progress, after all!) and it defers failure into an uncertain future (aka next work week). Right now, the project is under control and on its way. We can report 80% finished because we’ve done all the known parts. How hard can the unknown parts be anyway? Until they strike hard and wreck your estimates with “unforeseen challenges” and “sudden hardships”. At least this is what you tell your manager.

Risk first!

My preferred way to approach those projects is to reveal the whole map, to estimate all parts before I delve into the details. I already know most of the easy parts, but what about the unknown and/or hard parts? I don’t know their solution so I cannot reliably estimate their size. So I sit down and try to extract the core problem that I don’t know how to solve yet. This is the thing that prohibits an estimate. This is the white area on my map. This is the “here be dragons” area. If I spend my resources doing all the work other than this, I will succeed until I stand on the border of this area and see the dragon. And I will not have sufficient resources left. My allies (like my manager and colleagues) will grow weary. I will have to fight my hardest battle in the most inconvenient setting.

My approach is to take the risk upfront. Tackle the core problem and fail. Get up and tackle it again. Fail once more. And again. If you succeed with your task, the war is won. Your project will still require work, but it’s the easy kind of work (“just work”). You can estimate the remaining tasks and even if you’ve overspent in your first battle, you reliably know how much more resources you will need.

Fail fast

And if you don’t succeed? Well, then you know it with the least damage done. Your project will enter crisis mode, but in a position when there is still time and resources left. This is the concept of “fail fast”. To be able to fail fast, you need choose the “risk first” approach of task selection. To tackle the risk first, you need to be able to quantify the “risk” of your upcoming work.

Assessing risk

There are whole books about risk assessment that are interesting and helpful, but as a starter, you only need to listen to your stomach. If your stomach tells you that you are unsure about a specific part of your project, put that part on the “risky” list. If you don’t have a reliable stomach, try to estimate the part’s size. Do the estimation game with your colleagues. Planning poker, for example, is a great tool to uncover uncertainty because the estimates will differ. Just remember: Risk isn’t correlated with size. Just because a part is big doesn’t mean it is risky, too. Your crucial part can maybe be developed in an hour or two, given an inspiration and a cup of coffee.

Failing late means you’re out of options. Failing fast means you’ve eliminated an option and moved on.

Got issues? Treat them like micro-projects

Waterfall_modelEvery professional software developer organizes his work in some separable work tasks. These tasks are called issues and often managed in an issue tracker like Bugzilla or JIRA. In bigger teams, there is a separate project role for assigning and supervising work on the issue level, namely the project manager. But below the level of a single issue, external interference would be micro-management, a state that every sane manager tries to avoid at all costs.

Underneath the radar

But what if a developer isn’t that proficient with self-management? He will struggle on a daily basis, but underneath the radar of good project management. And there is nearly no good literature that deals specifically with this short-range management habits. A good developer will naturally exhibit all traits of a good project manager and apply these traits to every aspect of his work. But to become a good developer, most people (myself included!) need to go through a phase of bad project management and learn from their mistakes (provided they are able to recognize and reflect on them).

An exhaustive framework for issue processing

This blog entry outlines a complete set of rules to handle an work task (issue) like a little project. The resulting process is meant for the novice developer who hasn’t established his successful work routine yet. It is exhaustive, in the sense that it will cover all the relevant aspects and in the sense that it contains too much management overhead to be efficient in the long run. It should serve as a starting point to adopt the habits. After a while, you will probably adjust and improve it on your own.

A set of core values

The Schneide standard issue process was designed to promote a set of core values that our developers should adhere to. The philosophy of the value set itself contains enough details to provide another blog entry, so here are the values in descending order without further discussion:

  • Reliability: Your commitments need to be trustworthy
  • Communication: You should notify openly of changes and problems
  • Efficiency: Your work needs to make progress after all

As self-evident as these three values seem to be, we often discuss problems that are directly linked to these values.

The standard issue process

The aforementioned rules consist of five steps in a process that need to be worked on in their given order. Lets have a look:

  1. Orientation
  2. Assessment
  3. Development
  4. Feedback
  5. Termination

Steps three and four (development and feedback) actually happen in a loop with fixed iteration time.

Step 1: Orientation phase

In this phase, you need to get accustomed to the issue at hand as quickly as possible. Read all information carefully and try to build a mental model of what’s asked of you. Try to answer the following questions:

  • Do I understand the requirements?
  • Does my mental model make sense? Can I explain why the requirements are necessary?
  • Are there aspects missing or not sufficiently specified?

The result of this phase should be the assignment of the issue to you. If you don’t feel up to the task or unfamiliar with the requirements (e.g. they don’t make sense in your eyes), don’t accept the issue. This is your first and last chance to bail out without breaking a commitment.

Step 2: Assessment phase

You have been assigned to work on the issue, so now you need a plan. Evaluate your mental model and research the existing code for provisions and obstacles. Try to answer the following questions:

  • Where are the risks?
  • How can I partition the work into intermediate steps?

The result of this phase should be a series of observable milestones and a personal estimate of work effort. If you can’t divide the issue and your estimate exceeds a few hours of work, you should ask for help. Communicate your milestones and estimates by writing them down in the issue tracker.

Step 3: Development phase

You have a series of milestones and their estimates. Now it’s time to dive into programming. This is the moment when most self-management effort ends, because the developer never “zooms out” again until he is done or hopelessly stuck. You need periodic breaks to assess your progress and reflect on your work so far. Try to work for an hour (set up an alarm!) and continue with the next step (you will come back here!). Try to answer the following questions:

  • What is the most risky milestone/detail?
  • How long will the milestone take?

The result of this phase should be a milestone list constantly reordered for risk. We suggest a “cover your ass” strategy for novices by tackling the riskiest aspects first. After each period of work (when your alarm clock sets off), you should make a commit to the repository and run all the tests.

Step 4: Feedback phase

After you’ve done an hour of work, it’s time to back off and reflect. You should evaluate the new information you’ve gathered. Try to answer the following questions:

  • Is my estimate still accurate?
  • Have I encountered unforeseen problems or game-changing information?
  • What crucial details were discovered just yet?

The result of this phase should be an interim report to your manager and to your future self. A comment in the issue tracker is sufficient if everything is still on track. Your manager wants to know about your problems. Call him directly and tell him honestly. The documentation for your future self should be in the issue tracker, the project wiki or the source code. Imagine you have to repeat the work in a year. Write down everything you would want written down.
If your issue isn’t done yet, return to step three and begin another development iteration.

Step 5: Termination phase

Congratulations! You’ve done it. Your work is finished and your estimation probably holds true (otherwise, you would have reported problems in the feedback phases). But you aren’t done yet! Take your time to produce proper closure. Try to answer the following questions:

  • Is the documentation complete and comprehensible?
  • Have you thought about all necessary integration work like update scripts or user manual changes?

The result of this phase should be a merge to the master branch in the repository and complete documentation. When you leave this step, there should be no necessity to ever return to the task. Assume that your changes are immediately published to production. We are talking “going gold” here.

Recapitulation

That’s the whole process. Five steps with typical questions and “artifacts”. It’s a lot of overhead for a change that takes just a few minutes, but can be a lifesaver for any task that exceeds an hour (the timebox of step three). The main differences to “direct action” processes are the assessment and feedback phases. Both are mainly about self-observation and introspection, the most important ingredient of efficient learning. You might not appreciate at first what these phases reveal about yourself, but try to see it this way: The revelations set a low bar that you won’t fall short of ever again – guaranteed.