Category: Continuous Learning

Keeping it simple is hard indeed

A true story I was reminded of when I read Chad LaVigne’s essay on simplicity from the book “97 Things Every Software Architect Should Know”.

When I read the great book (or collection of essays) “97 Things Every Software Architect Should Know”, there was a story I had experienced first-hand for nearly every chapter. But it was only after I read about it in the well-placed words of somebody with greater in-depth experience than me that it appeared clear to me. The essence of my own experience was written down so I could iterate over it. Here is a story about simplicity that I hope will help somebody out there iterate over the same thought sometimes.

The problem

In the early days, we were asked to provide custom software for a nearly robot-like machine that performs measurements. Our software had to control some engines that could move sensors and actuators around. The whole machine was built by another company that only had eyes for the mechanical and electrical aspects. As a result, the communication protocol between the microcontrollers of the machine and our software was horribly awkward and unpleasant in regard of the software engineering side of the project.

One engine was the so-called “vertical engine”, because it could move an array of sensors up and down. There were four actual positions where the engine would stop automatically upon contact. It was the job of our software to send the correct commands in the correct order at the correct time to reach the destination. Effectively, we would “hop” from position to position, either moving the sensors up or down. If only the communication protocol would’ve allowed that. What we got was two commands: moving one position down and moving up to the topmost position. The movement options are depicted here:

The first solution

But what would a developer’s life be without a few challenges? Our team quickly came up with several possible solutions, from finite state machines to object graphs of “position” instances that offered several “movement options” to an agent-like object travelling inside the graph. In the brainstorming session, even more sophisticated possibilities were discussed. The solution we finally implemented was complex in terms of clarity. We needed quite a few unit and integration tests to proof the whole thing correct. During all this process, nobody ever questioned the complexity of the problem.

The essay

The chapter I read in the book “97 Things Every Software Architect Should Know” when this story came back to me was “Make Sure the Simple Stuff Is Simple” written by Chad LaVigne. Let me cite some sentences to show the message of his essay:

“People who design software are smart – really smart. The simple problem-complex solution trap can be an easy one to fall into because we like to demonstrate our knowledge. If you find yourself designing a solution so clever that it may become self-aware, stop and think. Does the solution fit the problem?”

The essay hit the nail on the head for me. Our solutions for this “vertical engine” were all clever and solved the problem, but a much more complex problem than the one that was really needed.

The second solution

Our first version of the software went into production and worked a treat. Years later, the hardware (specifically the microcontrollers) fell apart and was replaced by more sophisticated electronics. We needed to rewrite parts of our software, especially the engine controls. But this time, we had real movement patterns of all engines, collected in the logfiles. Some regular expression magic later (remember, we are smart people and like to show that!), it was clear that the vertical engine had never used two of the possible movement options. The real usage pattern of the engine is depicted in the picture below:

The real problem we should have solved from the beginning is much simpler than the original one. Every engine position has only one successor, there is no need to choose from several options or to calculate the shortest path to the destination. It’s the most boring finite state machine you’ve ever seen and there is no need for recursive structures.

It was our own cleverness that made the problem appear more complex than it really was, just because we could handle it. Some investigation on the problem domain instead of the solution domain would have brought us to the same conclusion that the log file analysis finally revealed.

The lesson learnt

The whole story isn’t about failure. Both solutions worked for the customer. In fact, he never noticed a difference when the second solution was going live. This story is about accidental complexity and about being smarter than the requirement. It’s in our nature to accept a problem as “given” and come up with an elegant solution. It might be much more effective to question the problem instead.

Thank you, Chad, for your great essay!

Follow-up to our Dev Brunch April 2010

A follow-up to our April 2010 Dev Brunch, summarizing the talks and providing bonus material.

Last sunday, we held our Dev Brunch for April 2010. It was the start of the open air season, introducing our new office roof garden. We brunched under a clear, sunny sky (no clouds, even no vapor trails from airplanes, as they aren’t allowed to fly because of some distant volcano ash emission) and talked about agile processes and books. A major part of the brunch was graded as “informal chatter”, just as it should be like.

The Dev Brunch

If you want to know more about the meaning of the term “Dev Brunch” or how we implement it, have a look at the follow-up posting of the brunch in October 2009. We continue to allow presence over topics. Our topics for the brunch were:

  • Agile life planning – Your software development daily routine is to incrementally build software from a backlog and release it after a fixed timebox. What might happen if you transfer this process to your private life (often called “real life”)? You’ll release a week’s life every week and have a week planning meeting on sunday, filling your week/sprint backlog with the most important items of your life’s backlog. Telling from this first hand experience of about a year, it works exceptionally well, improving the leisure time quality and making “progress” visible even at harder times. The only remaining question is who acts as the product owner.
  • Converting to Scrum – First hand experience of a team that, after years of “alarm call”-style development, successfully implemented Scrum as their primary process. The effects were at least less overtime, improved progress tracking, improved code quality (though Scrum is  only a management process!) and less pressure in the project. The customer even adjusted their wording when talking about new features: “next sprint” instead of “immediately, now”. The implemented process isn’t vanilla Scrum, but works nevertheless.
  • Defending Continuous Integration – What if, after an initial phase of excitement over the new tool, the continuous integration server really reveals flaws in your project? There seems to be the tendency to kill the messenger: Shut down the CI server and everything’s fine again. This talk was about the reasoning of both sides and some basic insights gained about machine sharing. Tip of the talk: Reserve your CI server exclusively for this task.
  • Book recommendations – A random collection of technical books read in the last half year, presented with a short summary and personal rating. Titles included were Coders At Work, Founders At Work, 97 Things Every Software Architect Should Know, 97 Things Every Project Manager Should Know, 97 Things Every Programmer Should Know, Hiring the Best Knowledge Workers, Techies & Nerds and Manage Your Project Portfolio. Yes, there are some patterns visible in the book titles. And it’s a good idea to keep some checklist of read articles for most of these books.
  • Dreyfus Model of Skill Acquisition – The Dreyfus Model differentiates five to six different stages of learning some arbitrary skill. It makes assumptions how the members of the stages work, how they process feedback information and what they need to get better. See the comment section for more information and bonus material.

As usual, the topics ranged from first-hand experiences to literature research. For additional information, check out the comment sections. Comments and resources might be in german language.

Retropection of the brunch

Holding the brunch in the bright morning sun, surrounded by rooftops and birds, really is a huge gain for the ambience factor. We even found a solution to produce fresh coffee up there. This will be a fun summer for the Dev Brunch.

Blog harvest, March/Easter 2010

Some noteworthy blog articles, harvested for March/Easter 2010. You’ll hear a lot about common software bugs, Java, inappropriate hiring procedures and Scala. If you ever wanted to know about the Fussy Bird in Scala, this blog post is for you, too.

Easter times means springtime, an hour stolen by the daylight saving time, rapidly changing weather and the first days without gloves (I catch chillblains very quick. There isn’t an adequate word for them in the german language, so I couldn’t name it for years). We are looking forward for a very hot summer, not only with the weather. But now, let’s talk about software.

  • Top 10 Web Software Application Security Risks – The OWASP community has released their current list of web programmer gotchas. There is nothing to add but “don’t try this at work, kids!”. For a broader audience, there’s the updated “Top 25 Most Dangerous Programming Errors” from the Common Weakness Enumeration project.
  • The 10 most common mistakes made in software development – Another top list with developer mistakes, written by Peter Horsten from GOYELLO IT Services. This one isn’t too code centric and doesn’t provide many solutions, but there is much wisdom in top 9 and 10.
  • I Have No Talent – There is a saying that open source development will make you humble, but John Nunemaker takes it to an extreme. He speaks true words. My favourite phrase is the last one, saying that he doesn’t compare himself to others, but to his own progress. If only everybody would do.
  • Why I love everything you hate about Java – An interesting blog entry by Nick Kallen. He takes a stand for the “bloated” middle-layer in (java) code. The post tends to be a long read, but I found it worthwile. The one thing I hate about Java is the slow progress in recent years.
  • Why I dropped Scala in favor of Java? – While you might argue about the universal importance of Subhash Chandran’s reasoning, it’s still valid. Combined with the previous blog harvest entry, there’s a pattern to be discovered: the grass isn’t always greener on the other side. But you have to change sides to see it.
  • NIO.2 : The new Path API in Java 7 – Java makes slow progress in recent years, but there is progress to be seen. The Path API is something we could have used years earlier. The DirectoryStream and WatchService are two functionalities you couldn’t easily build on your own yet.
  • Ant 1.8 Scanning Leaves 1.7.1 in the Dust – If you are using Ant, you can regain performance with just an upgrade. And you will find the other new features (like lexically scoped properties) really useful.

This was the more serious part of this harvesting. Let’s read some articles that share their message in a lighter way:

Finally, I want to introduce a “interesting tool” section. Each new harvest should present a framework/product/project I found interesting enough to tell you about it:

  • op4j – It calls itself a developer happiness tool. You may call it a stress test for static imports, collection kung-fu or fluency madness. But nevertheless, it’s fun to apply it to your boring old data structures.

P.S.: No easter egg is to be found in this posting.

Gesture Touchscreens might render Paper Prototyping useless

With the advent of highly dynamic, gesture-controlled user interfaces for touchscreens, Paper Prototyping seems to lose its applicability.

Paper Prototyping is an highly effective tool to examine the usability of software, even before it is written. The basic idea of Paper Prototyping is that you perform real (software) tasks with real users, but replace everything technical with low-fi substitutes.

Adventures in Low-fi

A typical Paper Prototyping session looks like this:

  • The user gets his task description and has to perform it with the software
  • The computer screen is replaced by an arrangement of paper sheets on a desk
  • The graphical user interface is replaced by hand-drawn copies on paper
  • The computer itself is replaced by a human, mimicking the software responses to input
  • The user operates by finger-pointing or writing with a pen

Advantages of Paper Prototyping

The whole situation described above seems awkward on first look, but is really rewarding for a project in its early stages. The customer has to provide real end user tasks and enough details of the solution to make up a prototype. The team has to produce reasonable drafts of the software GUI and come up with enough understanding of the processes and tasks involved to survive the session without major outages.

The result of a Paper Prototyping session can be used in various ways:

  • Detailed specification of the GUI
  • Use Case or User Story (acted out already)
  • Mock screenshots for the user manual
  • Data for initial acceptance tests

Classical user interface vs. gestures

This approach worked very well as long as the user only had a mouse (pointing, clicking) and a keyboard (writing) at hands. Even then, advanced features like grabbing (drag&drop) or automatic scrolling challenged the creativity of the prototypers. But most GUIs were rather dull and static. The perfect playground for Paper Prototyping.

With the advent of touchscreens, we soon realized that pointing and clicking only needs one finger out of ten. Gestures were introduced to keep all our fingertips busy and to enrich the interaction between user and GUI. We instantly understand the “zoom in” or “scroll down” gestures because it resembles natural behaviour (at least for some of us).

In the wake of gestures, the GUI of our software gets more and more dynamic. The GUI has to be minimalistic so we can control it even with stubby fingers (the new handicap of our generation, compare cell phone key pads). Detailed information has to be provided on demand and only temporarily. Everything can be manipulated. The classical approach to tab through a form (by carefully designed tab orders) isn’t that suitable anymore.

Gestures vs. Paper Prototyping

When using a Paper Prototype, the throughput of scribbled paper is enormous even with the classical GUIs. The more dynamic some dialog is, the more different parts need to be prepared in various states and locations (depending on the fragmentation of the paper screen). With gestures on a touchscreen, the user needs to be able to express them on the screen. Most touchscreen interfaces heavily depend on the (simulated) physical interaction between the fingers and some drawn “objects” on the interface. This is the moment when Paper Prototyping falls short of resembling the real interaction. You just can’t fiddle that fast with all the paper shreds.

No solution yet

I observed this effect when performing a Paper Prototype workshop with my students. The interfaces with classical mouse/keyboard handling performed well in the sessions. Interfaces for touchscreens (iPhone apps were the big newcomer here) just didn’t work out well, especially when downsized to palm size. We weren’t able to come up with a viable solution to make Paper Prototyping perform again for touchscreens and gestures.

Any ideas out there?

Blog harvest, February 2010

Some noteworthy blog articles, harvested for February 2010. If you ever asked yourself about the personality of your web framework, you’ll find the link to the answer here.

After the move to the new office is nearly complete, work begins to normalize again. Here is the February blog harvest with a little more entries, as I wasn’t unable to read other blogs, but to write on our own blog. There are many fun articles this time that I found share-worthy, perhaps because they made me laugh even in harder times.

This was the more serious part of this harvesting. Let’s read some articles that share their message in a lighter way:

  • What kind of woman would your web framework be? – If you ever have to sell a new hot (web) framework to management, why not take this plausible approach? At least they could relate to what you are talking about.
  • It’s Not the Recession, You Just Suck – Ouch! That hurt. This is a wake-up call for everybody who likes to blame it on higher means. And it reminds me to hurry up with this blog entry and get back to work.
  • I test therefore I log bugs – Ever tried to explain “programming” to your grandparents? You’ll end in esoterics (“teaching machines to have dreams”) or in obviousnesses. This is a story about consensus on the latter.

This blog harvest closes with a video:

  • Uncle Bob on Software Craftsmanship – Much of what Bob Martin says has truth in it, but for me the last two minutes are the most explicit and rewarding. By the way, Uncle Bob looks good in the T-Shirt (I always feared it would be teared, regarding the sounds when he stretches), but needs to switch his cell phone off.

Follow-up to our Dev Brunch February 2010

A follow-up to our February 2010 Dev Brunch, summarizing the talks and providing bonus material.

Today, we held our second Dev Brunch for 2010. It was the first one in the new office, with some packing cases still around. The brunch had some interesting topics, most of them small and focussed. We discussed if the topics should be announced beforehands to avoid collision, but defined these collisions as enrichments rather than duplications.

The Dev Brunch

If you want to know more about the meaning of the term “Dev Brunch” or how we implement it, have a look at the follow-up posting of the brunch in October 2009. This time, we didn’t urge all participants to bring their own topic. Presence is more important than topic.

  • Scrum adventure book review – There are lots of book on the Scrum project management process. But the one called “Geschichten vom Scrum” (sorry, it’s a german book!) will teach you all the basics and some advanced practical topics of Scrum while telling you the fairy tale of a kingdom haunted by dragons. By following a group of common fairy tale characters in their quest to build a dragon trap the Scrum way, you’ll learn a great share of real world Scrum and still be entertained. You might compare this book to Tom DeMarco’s “The Deadline”, a novel about general project management.
  • What is the Google Web Toolkit? – Based on the learning from the presentation of the Karlsruhe Java User Group (JUG-KA), we skipped through the slides to get to know the Google Web Toolkit (GWT) framework. Advanced topics were discussed in the next talk.
  • First hand experience with GWT – We talked about the sweet spots and pain points of Google Web Toolkit, based on the experiences in a real project. This was very helpful to sort out the marketing promises from the definite advantages. While the browser doesn’t affect the developer anymore, the separation of client (browser) and server will still leak through.
  • First impressions of the Lift framework – The way to go with web application development in Scala is Lift. It’s a framework borrowing the best from “Seaside, Rails, Django and Wicket” and combining it with Scala and the whole Java ecosystem. While this talk was just a teaser, it already looked promising.

As usual, the topics ranged from first-hand experiences to literature research or summaries of recently attended presentations. You can check out the comments for additional resources, but they may be in german language.

Retrospection of the brunch

It’s right to grant access to “non-topics”. This will lower the barrier for occasional guests while they are valuable for their experiences and insights. This brunch was enriched by yet another topic collision, which is the perfect situation for a more in-depth discussion.

Follow-up to our Dev Brunch January 2010

A follow-up to our January 2010 Dev Brunch, summarizing the talks and providing bonus material.

Today was our first Dev Brunch for the new year 2010. We held a well-attended and very interesting session with lots of coffee. It was the last brunch in the old office, as we are currently moving to new rooms. The brunch ended with a sneak peek into the new office.

The Dev Brunch

If you want to know more about the meaning of the term “Dev Brunch” or how we realize it, have a look at the follow-up posting of the brunch in October 2009. We used notebooks throughout the sessions today.

The topics of this session were:

  • Agile done wrong – A project that was converted to be agile now tends to be even more conservative when management lost faith in their developers. A rather sad first-hand story, with lots of Dilbert-style humor in it.
  • Implicits in Scala – Scala introduces a powerful feature of implicit (hence the name) type conversion that can be used to greatly simplify work with complex type systems. Or to totally disturb your understanding of it.
  • Follow-up on the local XP-Days – The XP Days Germany of andrena objects ag are a small, yet powerful conference in Karlsruhe. We got a summary of the overall style and different presentations. Things like Pokens, Pecha Kucha (watch your pronunciation of it) and live code katas are all very promising stuff. Most presentation content itself was interesting, too.
  • Exception safety in Java – A classical topic of (not only) C++, ported to Java. This overview presentation highlighted the basics of exception safety and some insights for Java, mostly borrowed from Alan Griffiths.
  • Preview of an Eclipse based product – We won’t go into much detail here, but we got a glance of an upcoming product that will greatly ease the use of multi-site programming with Eclipse. The EclipseCon 2010 in March might get promising.

The topics ranged from first-hand experiences to literature research. We look forward to provide additional information linked in comments on this article, partially in german language.

Retrospection of the brunch

It was very entertaining to meet everyone after the long holiday season. Lots of news and chatter and stuff. The topics were interesting and thought provoking. If you weren’t there, you’ve missed something. Check out the comments for compensation.

Blog harvest, January 2010

Some noteworthy blog articles, harvested for January 2010. Don’t miss the worst gadget gallery!

Welcome to the new year. We started late to work again this year and will collect some overtime in the next weeks as we are in the process of moving to a new office. This might impact the additional blog entries, so here is a quick blog harvest. Don’t miss the anti-gadget gallery at the end – but beware, it will make you laugh out loud, so don’t read it at work (unless you are supposed to have fun at work).

  • Reasons to Use Google Collections – Recently, the google collections library went gold. It’s a really nice collection of… collections. If you haven’t met it yet, do it now. You might read James Sugrue’s teaser alongside.
  • How Programming Books Promote Code Smells – Finally, somebody shares my opinion on “code examples crippled for brevity”. I got disgusted by some java books because their code examples were so bad, it hurt. The situation eases a lot with the availability of “real” code in open source projects that you can read as recompense.
  • Test-Driven Teaching! – Another problem with bad code examples is to avoid them in live examples (like in lectures). Here is an interesting article by Peter Karich presenting the idea of integrating test code in your lectures. I’ll see if my students like the idea.
  • A performance tuning story – The best stories are real stories. This story has all that’s needed: a mysterious effect at the beginning, fast-paced action in the middle and an open end. The X files weren’t as thrilling as this one.
  • Maven and Ant guys: you’ll never agree. On anything. Period. Deal with it! – What would we do if we couldn’t join an eternal flame war at surf time? You can join on one side (there are always two sides, seldom more!) or stand in between and pick on both. Lieven Doclo chose the Maven side of builds.
  • Maven Mythbusters – The most prominent maven myths get busted by John Ferguson Smart to end the flame war mentioned above once and for all (or at least take out some misinformation). I’ve linked to the first busted myth, you’ll have no trouble finding the other(s). Read the comments, too!
  • The Top 10 Posts of this Blog Over the Years – Stephan Schmidt’s blog “Code Monkeyism” is a regular link target in my harvests. When he harvested his own blog for the most popular entries, I couldn’t resist linking him once more. Maybe this is called meta-harvesting.

This was the article side of this harvesting. Let’s continue to have fun by sliding through the gallery and then… sliding through your repository:

  • Gizmodo’s Worst Gadget Gallery – Oh yes, ten years full of technological crap, complete with pictures and a short description. If you spend 30 seconds on each gadget, you’re in for half an hour pure fun. My favorites are the DeathStar for personal reference (we lost 7 of 9) and the Eye-Track for its funny description.
  • Mining your source code repository – Part 1 – Yes, all your legacy repositories are still there. Discover the archeologist in yourself and perform some data mining on them. This article may be the starting point for your new career.

Booked in February

A short book preview of the upcoming O’Reilly title “97 Things Every Programmer Should Know” and our participation in it.

Ok, the title is a bit misleading – it’s a play of words(*). This entry is actually a book preview on the upcoming book “97 Things Every Programmer Should Know” from O’Reilly.

97 Things Of Wisdom

The “97 Things” series started out with “97 Things Every Software Architect Should Know” early last year. The book essentially is a collection of short articles on specific topics that should bother today’s software architect. You may classify as a software architect if you don’t just stir up source code but are also in charge to give the system a shape.

The articles are straight to the point and can be read within five minutes each. Don’t expect detailed textbook chapters of the topics, but they work extremely well as creative appetizers. And there are nearly a hundred appetizers from well-respected members of the software architect community in this book.

Just imagine you would meet all the authors for five minutes each on a conference and just ask them for an appealing thought. This book serves as the best replacement for it.

Wisdom continued

Soon after the first book, there was a second book in the series, “97 Things Every Project Manager Should Know”. I haven’t read this one yet, but it is on my must-read list for 2010.

And now, next month, there will be another book, this time for the fellow coder: “97 Things Every Programmer Should Know”. As usual, there are 97 selected articles with bits of wisdom from big community names. Kevlin Henney is the editor for this book (we featured him in our last blog harvest). You can take a sneak peek online in the 97TEPSK wiki, where the articles were fostered (and a second part is likely to emerge). But don’t forget to buy a paper copy that you can foist on your peers to inspire them, too.

Telling from the articles I’ve read so far, the book will be great. Please don’t expect detailed language specifics, lengthy code examples or fancy UML diagrams. But expect a whole bunch of great ideas that stem from real experiences of real programmers.

One percent of a book

What’s our relation to the new book? We’ve contributed an article to it! Even if we thereby only wrote approximately one percent of the book, it feels great and we consider ourselves honored.

The topic of our article is Extreme Feedback Devices (XFD): “Let Your Project Speak for Itself”. We gathered quite a lot of these devices over the years and ran a few experiments, so we thought we are qualified to write about it. And there it is, the first bit of our wisdom, printed in a book.

We will, of course, continue to publish our wisdom on this blog first. If you’ve followed us over the last years, the article comes as no real surprise. But I’m sure some other articles of the book will. Go buy it!

(*) Play of words in a language other than your native tongue are always dangerous. I hope this one worked out well.

A blind spot of Continuous Integration

Imagine you haven’t changed a continuously integrated project for months when suddenly a unit test breaks. Here’s why CI has a blind spot with flawed tests.

In the early days of April 2008, we updated our hudson continuous integration (CI) server to a new version. This was no unusual action, as there was a new version every day back then, bringing new features in a rapid rate. What was unusual after the upgrade was that one of the surveilled projects failed to build all of a sudden.

Sudden (test) failure without a change

The build was started manually, without a code change. The project itself was inactive back then, meaning that no changes were made for months. And suddenly, a unit test broke. The test was in the project for two whole years without ever going off. What happened?

Good unit tests

There are rules for good unit tests. A basic set are the A-TRIP rules formulated in the excellent beginner’s book “Pragmatic Unit Testing” by Andy Hunt and Dave Thomas. The failed test clearly disobeyed the “repeatable” rule (the R in A-TRIP): It didn’t result in the same result as before while the code under test didn’t change.

Write repeatable tests or your CI will be blind (partially)

The cause of the failed test was putting the clocks back because the daylight-saving time part of the year began. The unit test secured some date calculations by taking the current date and comparing it to future and past dates that got calculated. The calculation went wrong when the daylight-saving mode changes in the calculated period, which was a bug. Repeatability of the unit test was lost when “the current date” entered the code – whether on the unit test or productive code side.

Two years of blindness

How could this bug survive two years without being noticed? The project was under CI surveillance since the beginning, the unit test to detect the bug was present along with the bugged code. The answer is: We never programmed for this project around the weeks of the year when the clock is adjusted and the bug occurs. This was a coincidence influenced by the customer’s schedule. So every time CI (or we) ran the unit test, it passed. Until that day right after putting the clocks back.

How to avoid this blind spot

There are two things you can do to avoid this scenario:

  • Always inject a fixed “the current date” into your code when dealing with date calculations. Only use absolute dates in your unit tests. Time isn’t a healer for your tests, it’s a beast to be tamed.
  • Set up a nightly build for your project that runs once a day even when no changes have been made. It would have caught this bug one and a half year earlier.

To sum it up:

CI only spots bugs when they move (aka the code is changed). Nightly builds provide a (fuzzy) security layer against non-repeatable unit tests. And unit tests with flaws provide only delusory security.

Additional background information

After fully understanding the circumstances, we were curious why the customer didn’t notice the bug and asked him about it. The answer was delightful: “Our computers don’t adjust their clocks. Daylight-saving time only causes trouble.” What a wise decision!

For a good comparison of CI vs. Nightly Builds see this blog entry.