Chopping up big tasks

As a programmer, you have probably dealt with a task that seemed simple enough in the beginning but just keeps going and going and going. I have been chewing on such a task for the better part of the last three weeks and finally closed it today*. A really long time, considering I usually complete my tasks in less than a day, up to three days for especially long ones.

I know that many programmers can get lost with big tasks like this. I am certainly no exception. Analysis paralysis and decision fatigue get easily get the best of you, considering the mountain of work still ahead.

But I have a few ways to deal with such situations. Of course, your milage may vary. But I am sure that without them, this specific issue would have taken me even longer. It boils down to one rule:

Focus on the essentials only.

This obviously relates to yak shaving. Sometimes you need to do something else first before you can complete your task. This is recursive and can quickly take up lots of time. This will ultimatly be required, but for the moment it distracts from the original task. While you complete a side task, the main task will not advance, leading to a feeling of getting stuck, technical problems (like merge conflicts in long-running branches) and psychological problems (like decision fatigue).

So what can you do about this? My advice is to rigorously cut-off side tasks, by taking up technical debt temporarily. I annotate my code with HACK, TODO and FIXME to mark all the isolated spots I still need to change for the 100% version. The end feature (= user story) does not have to be completed by the end of my task, but I should be reasonably confident that the main work is done. Anything to that end will work.

Sometimes single isolated spots will not be enough for a side-task. In that case, I will usually mark it with an issue number from our issue-tracker right away.

After completing the main work in this way, but before I merge my code or close my original work-item/issue, I make another pass over all the HACK, TODO and FIXMEs I generated. The smaller ones I fix right away. Anything where the way to complete them is not super obvious gets converted into an issue in the issue tracker, and cross-linked from the code. This means I add a comment referencing the issue from the code and I make sure that the issue says that it is marked in the code. E.g., for this specific task, I now have 6 open follow-up issues.

After that, I usually merge the code into the main branch. If it’d break something or be misleading with all the follow up issues not done yet, the feature can sometimes be disabled with a feature toggle. Alternatively, the follow up tasks can be completed in their own branches and merged back onto the main task’s branch, which can be merged once everything is done. This hugely depends on your product cycle, of course.

Do you have any clever methods to handle bigger tasks?

Got issues? Treat them like micro-projects

Issues should be the smallest work unit available. But what if it is still larger than you can manage? Here’s a standard process for self-management while working on an issue.

Waterfall_modelEvery professional software developer organizes his work in some separable work tasks. These tasks are called issues and often managed in an issue tracker like Bugzilla or JIRA. In bigger teams, there is a separate project role for assigning and supervising work on the issue level, namely the project manager. But below the level of a single issue, external interference would be micro-management, a state that every sane manager tries to avoid at all costs.

Underneath the radar

But what if a developer isn’t that proficient with self-management? He will struggle on a daily basis, but underneath the radar of good project management. And there is nearly no good literature that deals specifically with this short-range management habits. A good developer will naturally exhibit all traits of a good project manager and apply these traits to every aspect of his work. But to become a good developer, most people (myself included!) need to go through a phase of bad project management and learn from their mistakes (provided they are able to recognize and reflect on them).

An exhaustive framework for issue processing

This blog entry outlines a complete set of rules to handle an work task (issue) like a little project. The resulting process is meant for the novice developer who hasn’t established his successful work routine yet. It is exhaustive, in the sense that it will cover all the relevant aspects and in the sense that it contains too much management overhead to be efficient in the long run. It should serve as a starting point to adopt the habits. After a while, you will probably adjust and improve it on your own.

A set of core values

The Schneide standard issue process was designed to promote a set of core values that our developers should adhere to. The philosophy of the value set itself contains enough details to provide another blog entry, so here are the values in descending order without further discussion:

  • Reliability: Your commitments need to be trustworthy
  • Communication: You should notify openly of changes and problems
  • Efficiency: Your work needs to make progress after all

As self-evident as these three values seem to be, we often discuss problems that are directly linked to these values.

The standard issue process

The aforementioned rules consist of five steps in a process that need to be worked on in their given order. Lets have a look:

  1. Orientation
  2. Assessment
  3. Development
  4. Feedback
  5. Termination

Steps three and four (development and feedback) actually happen in a loop with fixed iteration time.

Step 1: Orientation phase

In this phase, you need to get accustomed to the issue at hand as quickly as possible. Read all information carefully and try to build a mental model of what’s asked of you. Try to answer the following questions:

  • Do I understand the requirements?
  • Does my mental model make sense? Can I explain why the requirements are necessary?
  • Are there aspects missing or not sufficiently specified?

The result of this phase should be the assignment of the issue to you. If you don’t feel up to the task or unfamiliar with the requirements (e.g. they don’t make sense in your eyes), don’t accept the issue. This is your first and last chance to bail out without breaking a commitment.

Step 2: Assessment phase

You have been assigned to work on the issue, so now you need a plan. Evaluate your mental model and research the existing code for provisions and obstacles. Try to answer the following questions:

  • Where are the risks?
  • How can I partition the work into intermediate steps?

The result of this phase should be a series of observable milestones and a personal estimate of work effort. If you can’t divide the issue and your estimate exceeds a few hours of work, you should ask for help. Communicate your milestones and estimates by writing them down in the issue tracker.

Step 3: Development phase

You have a series of milestones and their estimates. Now it’s time to dive into programming. This is the moment when most self-management effort ends, because the developer never “zooms out” again until he is done or hopelessly stuck. You need periodic breaks to assess your progress and reflect on your work so far. Try to work for an hour (set up an alarm!) and continue with the next step (you will come back here!). Try to answer the following questions:

  • What is the most risky milestone/detail?
  • How long will the milestone take?

The result of this phase should be a milestone list constantly reordered for risk. We suggest a “cover your ass” strategy for novices by tackling the riskiest aspects first. After each period of work (when your alarm clock sets off), you should make a commit to the repository and run all the tests.

Step 4: Feedback phase

After you’ve done an hour of work, it’s time to back off and reflect. You should evaluate the new information you’ve gathered. Try to answer the following questions:

  • Is my estimate still accurate?
  • Have I encountered unforeseen problems or game-changing information?
  • What crucial details were discovered just yet?

The result of this phase should be an interim report to your manager and to your future self. A comment in the issue tracker is sufficient if everything is still on track. Your manager wants to know about your problems. Call him directly and tell him honestly. The documentation for your future self should be in the issue tracker, the project wiki or the source code. Imagine you have to repeat the work in a year. Write down everything you would want written down.
If your issue isn’t done yet, return to step three and begin another development iteration.

Step 5: Termination phase

Congratulations! You’ve done it. Your work is finished and your estimation probably holds true (otherwise, you would have reported problems in the feedback phases). But you aren’t done yet! Take your time to produce proper closure. Try to answer the following questions:

  • Is the documentation complete and comprehensible?
  • Have you thought about all necessary integration work like update scripts or user manual changes?

The result of this phase should be a merge to the master branch in the repository and complete documentation. When you leave this step, there should be no necessity to ever return to the task. Assume that your changes are immediately published to production. We are talking “going gold” here.

Recapitulation

That’s the whole process. Five steps with typical questions and “artifacts”. It’s a lot of overhead for a change that takes just a few minutes, but can be a lifesaver for any task that exceeds an hour (the timebox of step three). The main differences to “direct action” processes are the assessment and feedback phases. Both are mainly about self-observation and introspection, the most important ingredient of efficient learning. You might not appreciate at first what these phases reveal about yourself, but try to see it this way: The revelations set a low bar that you won’t fall short of ever again – guaranteed.

Branching support missing in JIRA

JIRA is a really great tool when it comes to issue tracking. It is powerful, extensible, usable and widespread. We and many of our clients use it for years and are quite satisfied coming from other tools like Bugzilla.

One thing we find is missing though is the concept of branching. In JIRA you have projects and can define a roadmap consisting of versions that follow each other. Many software products require sooner or later a development line which is only maintained getting bug and security fixes while feature development happens on a separate branch.

Tracking issues for a software with different branches is a bit more demanding because you have to identify the branches one issue affects and possibly separately fix them on each branch. One and the same issue could have different resolutions per branch, i.e. invalid if a broken functionality does not exist anymore. Nevertheless, all branches have to be checked, likely in different time frames.

Unfortunately JIRA does not support the notion of branches. You have to emulate the behaviour using different schemes like:

  • One issue per version that represents a branch
  • Multiple fix versions for an issue
  • Subtasks for an issue or issue links to ckeck and fix the issue for different versions

To me all those are workarounds which lack polished usability and add overhead to your issue management. Real branching support could help you check on which branch an issue is done, where it has still to be resolved and so on without adding more and more (perhaps loosely connected) issues to your system or forgetting to fix the issue on some branch (no question/warning when resolving an issue with multiple fix versions).

[Update:]

I created a new feature request for JIRA where you can vote or track progress on this issue.