Author: daniel.lindner

A short story about priorities

When you mix your own priorities with the ones of your customers, embarrassing things might happen.

Before I can tell you the story, I have to add this disclaimer: The story is true, but it wasn’t us. To protect the identities, I’ve changed nearly all the details, hopefully without watering down the essence of the story.

The assignment

A few years ago, when developing software for mobile devices wasn’t as ordinary as it is today, a freelance developer got a call from a loyal customer. He should develop a little web application that could also be used on a smartphone. The functionality of the application was minimal: Show a neat image containing two buttons to download a song teaser in two different file formats, probably MP3 and WMA. That’s all. The whole thing was a marketing app that should be used a few weeks or months and then be mothballed. Being a marketing app, the payment was very decent.

The development

The freelancer was excited to say the least. This was his chance to enter the emerging field of mobile device software development. And because it should be a web application, he would try out all these new fancy mobile web frameworks and settle for the best. The image of the website should rotate and adjust to the device screen if the user tilted his phone. And because phone screens weren’t that well equipped with pixels as today and mobile internet was expensive, the image should be optimized for the best possible tradeoff between details and file size.

Some mobile web frameworks provided access to the device type, so there were all kinds of javascript magic tricks to be potentially applied. Well, they weren’t applied, but very well could have been. The freelancer prototyped most of them, out of curiosity.

The web application itself was completed in one session once the song files and the master image were provided by the customer. The freelancer installed everything on a staging web server and communicated back success. The customer was pleased by the quick reaction time and appointed a meeting to inspect the results.

The presentation

The customer gave the freelancer a notebook attached to a beamer to present the application. The notebook had a screen resolution many times the envisioned target platform, so the intro image was very pixellated and far from appealing. The freelancer used his smartphone to present the image on a smaller screen, but the customer only shook his head: “You should develop a web application for both phones and PCs. Most of our customers will use their PC to visit the site.” The freelancer apologized and promised to add another browser switch to present a full resolution image to PC users.

Now, the freelancer continued to present the web application on his phone. He invited the customer to tilt the device and was satisfied when the web site adjusted perfectly. The customer was pleased, too. Then, he tapped on one of the buttons. Nothing happened. The whole application consisted only of two buttons and one wasn’t working. The freelancer frantically tried to figure out what had gone wrong, when the customer tapped on the other button. No reaction from that one, too. “But I’ve uploaded both song files to the right location with the right access rights.”, the freelancer just said to himself when it dawned him: He forgot to insert the link tags in the HTML file. The buttons were just images. He never actually clicked on one of the buttons during development.

The moral of the story

Recapitulating, the freelancer was asked to develop a web application with an image and two download buttons, but he managed to cripple the image for the larger part of the anticipated users and never provide any download functionality at all. The customer’s requirements somehow got lost along the way.

This isn’t so much a story about a confused freelancer or improper requirement analysis. It’s a story about priorities. The customer expressed his priorities through the requirements. The freelancer superimposed his own priorities very early in the process (without telling anybody) and never returned to the original set until the presentation. And while it is granted to have a secondary agenda as a service provider, it should never interfere with the primary agenda – the one set by the customer.

Don’t fool yourself into thinking that this could never happen to you. It’s not always as obvious as in this story. Some, if not most of your customer’s priorities are unintentionally (or even intentionally) kept secret. They can only be traced during live exercises, which is why early prototyping and using the prototype in real scenarios is a good way to reveal them.

Summary of the Schneide Dev Brunch at 2012-05-27

If you couldn’t attend the Schneide Dev Brunch in May 2012, here are the main topics we discussed neatly summarized.

Yesterday, we held another Schneide Dev Brunch on our roofgarden. The Dev Brunch is a regular brunch on a sunday, only that all attendees want to talk about software development and various other topics. If you bring a software-related topic along with your food, everyone has something to share.

We had to do another introductory round because there were new participants with new and very interesting topics. This brunch was very well attended and rich in information. Let’s have a look at the main topics we discussed:

Agile wording (especially SCRUM)

This was just a quick overview over the common agile vocabulary and what ordinary people associate with them. A few examples are “scrum“, “sprint” and “master”. We agreed that some terms are flawed without deeper knowledge about the context in agile.

Book: “Please Understand Me”

if you are interested in the Myers-Briggs classification of personality types (keywords: are you INTJ, ESTP or INFP?), this is the book to go. It uses a variation of the personality test to classify and explain yourself, your motives and personal traits. And if you happen to know about the personality type of somebody else, it might open your eyes to the miscommunication that will likely occur sooner or later. Just don’t go overboard with it, it’s just a model about the most apparent personality characteristics. The german translation of the book is called “Versteh mich bitte” and has some flaws with typing and layouting errors. If you can overlook them, it might be the missing piece of insight (or empathy) you need to get through to somebody you know.

TV series: “Dollhouse”

As most of us are science fiction buffs and hold a special place in our heart for the series “Firefly”, the TV series “Dollhouse” by Joss Whedon should be a no-brainer to be interested in. This time, it lasted two seasons and brings up numerous important questions about programmability every software developer should have a personal answer for. Just a recommendation if you want to adopt another series with limited episode count.

Wolfpack Programming

A new concept of collaborative programming is “wolfpack programming” (refer to pages 21-26). It depends on a shared (web-based) editor that several developers use at once to develop code for the same tasks. The idea is that the team organizes itself like a pack of wolves hunting deer. Some alpha wolves lead groups of developers to a specific task and the hunt begins. Some wolves/developers are running/programming while the others supervise the situation and get involved when convenient. The whole code is “huntable”, so it sounds like a very chaotic experience. There are some tools and reports of experiments with wolfpack programming in Smalltalk. An interesting idea and maybe the next step beyond pair programming. Some more information about the editor can be found on their homepage and in this paper.

Book: “Durchstarten mit Scala”

Sorry for the german title, but the book in this review is a german introductory book about Scala. It’s not very big (around 200 pages) but covers a lot of topics in short, with a list of links and reading recommendations for deeper coverage. If you are a german developer and used to a modern object-oriented language, this book will keep its promise to kickstart you with Scala. Everything can be read and understood easily, with only a few topics that are more challenging than there are pages for them in the book. The topics range from build to test and other additional frameworks and tools, not just core Scala. This book got a recommendation for being concise, profound and understandable (as long as you can understand german).

Free Worktime Rule

This was a short report about employers that pay their developers a fixed salary, but don’t define the workload that should happen in return. Neither the work time nor the work content is specified or bounded. While this sounds great in the first place (two hours of work a week with full pay, anybody?), we came to the conclusion that peer pressure and intrinsic motivation will likely create a dangerous environment for eager developers. Most of us developers really want to work and need boundaries to not burn out in a short time. But an interesting thought nevertheless.

Experimental Eclipse Plugin: “Code_Readability”

This was the highlight of the Dev Brunch. One attendee presented his (early stage) plugin for Eclipse to reformat source code in a naturally readable manner. The effect is intriguing and very promising. We voted vehemently for early publication of the source code on github (or whatever hosting platform seems suitable). If the plugin is available, we will provide you with a link. The plugin has a tradition in the “Three refactorings to grace” article of the last Dev Brunch.

Light Table IDE

A short description of the new IDE concept named “Light Table”. While the idea itself isn’t new at all, the implementation is very inspirational. In short, Lighttable lets you program code and evaluates it on the fly, creating a full feedback loop in milliseconds. The effects on your programming habits are… well, see and try it for yourself, it’s definitely worth a look.

Inventing on Principles

Light Table and other cool projects are closely linked to Bret Victor, the speaker in the mind-blowing talk “Inventing on Principles”. While the talk is nearly an hour of playtime, you won’t regret listening. The first half of the talk is devoted to several demo projects Bret made to illustrate his way of solving problems and building things. They are worth a talk alone. But in the second half of the talk, Bret explains the philosophy behind his motivation and approach. He provides several examples of people who had a mission and kept implementing it. This is very valuable and inspiring stuff, it kept most of us on the edge of our seats in awe. Don’t miss this talk!

Albatros book page reminder (and Leselotte)

If you didn’t upgrade your reading experience to e-book readers yet, you might want to look at these little feature upgrades for conventional books. The Albatros bookmark is a page remembering indexer that updates itself without your intervention. We could test it on a book and it works. You might want to consider it especially for your travelling literature. This brought us to another feature that classic dead wood books are lacking: the self-sustained positioning. And there’s a solution, too: The “Leselotte” is a german implementation of the bean bag concept for a flexible book stand. It got a recommendation by an attendee, too.

Bullshit-Meter

If you ever wondered what you just read: It might have been bullshit. To test a text on its content of empty phrases, filler and hot air, you can use the blabla-meter for german or english text. Just don’t make the mistake to examine the last apidoc comments you hopefully have written. It might crush your already little motivation to write another one.

Review on Soplets

In one of the last talks on the Java User Group Karlsruhe, there was a presentation of “Soplets”, a new concept to program in Java. One of our attendees summarized the talk and the concept for us. You might want to check out Soplets on your own, but we weren’t convinced of the approach. There are many practical problems with the solution that aren’t addressed yet.

Review on TDD code camp

One of our attendees lead a code camp with students, targeting Test Driven Development as the basic ruleset for programming. The camp rules closely resembled the rules of Code Retreats by Corey Haines and had Conway’s Game of Life as the programming task, too. With only rudimentary knowledge about TDD and Test First, the students only needed four iterations to come up with really surprising and successful approaches. It was a great experience, but showed clearly how traditional approaches like “structured object-oriented analysis” stands in the way of TDD. Example: Before any test was written to help guide the way, most students decided on the complete type structure of the software and didn’t aberrate from this decision even when the tests told them to.

Report of Grails meetup

Earlier last week, the first informal Grails User Group Karlsruhe meeting was held. It started on a hot late evening some distance out of town in a nice restaurant. The founding members got to know each other and exchanged basic information about their settings. The next meeting is planned with presentations. We are looking forward to what this promising user group will become.

Epilogue

This Dev Brunch was a lot of fun and new information and inspiration. As always, it had a lot more content than listed here, this summary is just a best-of. We are looking forward to the next Dev Brunch at the Softwareschneiderei. And as always, we are open for guests and future regulars. Just drop us a notice and we’ll invite you over next time.

Summary of the Schneide Dev Brunch at 2012-03-25

If you couldn’t attend the Schneide Dev Brunch in March 2012, here are the main topics we discussed for you to review.

This summary is a bit late and my only excuse it that the recent weeks were packed with action. But the good news is: The Schneide Dev Brunch is still alive and gaining traction with an impressive number of participants for the most recent event. The Schneide Dev Brunch is a regular brunch in that you gather together to have a late breakfast or early dinner on a sunday, only that all attendees want to talk about software development (and various other topics). If you bring a software-related topic along with your food, everyone has something to share. We were able to sit in the sun on our roofgarden and enjoy the first warm spring weekend.

We had to do introductory rounds because there were quite some new participants this time. And they brought good topics and insights with them. Let’s have a look at the topics we discussed:

Checker Framework

This isn’t your regular java framework, meant to reside alongside all the other jar files in your dependency folder. The Checker framework enhances java’s type system with “pluggable types”. You have to integrate it in your runtime, your compiler and your IDE to gain best results, but after that you’re nothing less than a superhero among regulars. Imagine pluggable types as additional layers to your class hierarchy, but in the z-axis. You’ll have multiple layers of type hierachies and can include them into your code to aid your programming tasks. A typical use case is the compiler-based null checking ability, while something like Perl’s taint mode is just around the corner.

But, as our speaker pointed out, after a while the rough edges of the framework will show up. It still is somewhat academic and lacks integration sometimes. It’s a great help until it eventually becomes a burden.

Hearing about the Checker framework left us excited to try it sometimes. At least, it’s impressive to see what you can do with a little tweaking at the compiler level.

Getting Stuck

A blog entry by Jeff Wofford inspired one of us to talk about the notion of “being stuck” in software development. Jeff Wofford himself wrote a sequel to the blog entry, differentiating four kinds of stuck. We could relate to the concept and have seen it in the wild before. The notion of “yak shaving” entered the discussion soon. In summary, we discussed the different types of being stuck and getting stuck and what we think about it. While there was no definite result, everyone could take away some insight from the debate.

Zen to Done

One topic was a review of the Zen to Done book on self-organization and productivity improvement. The methodology can be compared to “Getting Things Done“, but is easier to begin with. It defines a bunch of positive habits to try and establish in your everyday life. Once you’ve tried them all, you probably know what works best for you and what just doesn’t resonate at all. On a conceptional level, you can compare Zen to Done to the Clean Code Developer, both implementing the approach of “little steps” and continuous improvement. Very interesting and readily available for your own surveying. There even exists a german translation of the book.

Clean Code Developer mousepads

Speaking of the Clean Code Developer. We at the Softwareschneiderei just published our implementation of mousepads for the Clean Code Developer on our blog. During the Dev Brunch, we reviewed the mousepads and recognized the need for an english version. Stay tuned for them!

Book: Making software

The book “Making software” is a collection of essays from experienced developers, managers and scientists describing the habits, beliefs and fallacies of modern software development. Typical for a book from many different authors is the wide range of topics and different quality levels in terms of content, style and originality. The book gets a recommendation because there should be some interesting reads for everyone inside. One essay was particularly interesting for the reviewer: “How effective is Test-Driven Development?” by Burak Turhan and others. The article treats TDD like a medicine in a clinical trial, trying to determine the primary effects, the most effective dosage and the unwanted side effects. Great fun for every open-minded developer and the origin of a little joke: If there was a pill you could take to improve your testing, would a placebo pill work, too?

Book: Continuous Delivery

This book is the starting point of this year’s hype: “Continuous Delivery” by Jez Humble and others. Does it live up to the hype? In the opinion of our reviewer: yes, mostly. It’s a solid description of all the practices and techniques that followed continuous integration. The Clean Code Developer listed them as “Continuous Integration II” until the book appeared and gave them a name. The book is a highly recommened read for the next years. Hopefully, the practices become state-of-the-art for most projects in the near future, just like it went with CI. The book has a lot of content but doesn’t shy away from repetition, too. You should read it in one piece, because later chapters tend to refer to earlier content quite often.

Three refactorings to grace

The last topic was the beta version of an article about the difference that three easy refactorings can make on test code. The article answered the statement of a participant that he doesn’t follow the DRY principle in test code in a way. It is only available in a german version right now, but will probably be published on the blog anytime soon in a proper english translation.

Epilogue

This Dev Brunch was a lot of fun and had a lot more content than listed here. Some of us even got sunburnt by the first real sunny weather this year. We are looking forward to the next Dev Brunch at the Softwareschneiderei. And as always, we are open for guests and future regulars. Just drop us a notice and we’ll invite you over next time.

Clean Code Developer at your fingertips

You’ve probably already heard about the Clean Code Developer initiative. We’re donating a full spectrum of mousepad designs for your educational support.

We are participants in the Clean Code Developer (CCD) movement. This initiative provides a way to perpetually learn, train, reflect and act on the most important topics of today’s software development by formulating a value system and a learning path. The learning path is subdivided in different grades, associated with colors. Every Clean Code Developer progresses continually through the grades, focussing on the principles and practices of the current grade.

If you want a tongue-in-cheek explanation of what the Clean Code Developer is in one sentence: It’s a sight-seeing tour to the most prominent topics every professional software developer should know. But other than your usual tourist rip-off, you can just stay seated and enjoy another round without ever paying anything except attention.

Visualize it

An important aspect of learning and deliberate practice is proper visualization. We invest a lot of work at our workplace, our software and the interaction with our customers to make things visible. When we reflected on our Clean Code Developer practice, we knew that it lacked visualization.

The proposed equipment for a Clean Code Developer is a desktop background picture, a mousepad with an image of all grades at once and some rubber wristbands in the colors of the grades. The wristbands serve as a reminder and a self-assessment tool. The desktop background picture is nice, but only visible if we don’t perform actual work. This let us concentrate on the mousepad.

Duplicate if necessary

The mousepad is the most prominent “advertising” space on the typical work desktop. We want to advertise the content of our current Clean Code Developer grade to ourself. The combination of these two thoughts is not one mousepad, but one for every grade. Imagine six mousepads in the colors of the grades, displaying your currently most important topics right under your fingertips.

We liked the idea so much that we worked on it. The result is a collection of mousepads for every Clean Code Developer to enjoy.

Iterative design

It took us several full cycles of planning, design, layout and proof-reading to have the first version of mousepads produced. It took only a few hours of real-world testing to start the second iteration to further improve the design. Right now, we are on the third iteration. The first iteration had the five colored CCD grades printed on real mousepads. The second iteration added the mousepad for the white grade and a little stand-up display for the initial black grade. The third iteration incorporated the official Clean Code Developer logo, the website URL and improved some details.

Here are some promotional photos of the five first-iteration mousepads:

This slideshow requires JavaScript.

As you can see, we chose to print ultra-slim mousepads to test if it’s feasible to use them stacked all at once (it isn’t, your mileage may vary) or use them even if you aren’t used to mousepads at all (it depends, really). You might want to print the images onto the mousepad you prefer best.

Do it yourself

Yes, you’ve read it right. We are donating the mousepad images back to the community. You can download everything right here:

All documents are bare of any company logo or other advertising and free for your constructive usage. There is only one catch really: the documents are in german language. This might not be apparent at first because we really like the original english technical terms, but some content might need translation for non-german speakers. If you are interested to produce an all-english version, drop us a line.

Acknowledgements

These mousepads wouldn’t exist without the help and inspiration of many co-workers. First of all, the founders of the Clean Code Developer movement, Ralf Westphal and Stefan Lieser, provided all the content of the mousepads. Without their groundbreaking work, we probably wouldn’t have thought of this. The design and production is owed to Hannegret Lindner from the Hannafaktur, a small graphic design agency. We admire her endurance with our iterative approach. And finally, the initial inspiration sparked in a creative discussion with Eric Wolf and Benjamin Trautwein from ABAS Software AG.

It’s your turn now

We are very curious about your story, photo or action still with the mousepads (or the little stand-up display). You can also just share your thoughts about the whole idea or submit an improvement. We’d love to hear from you.

Take your programming course with a grain of salt, please

If you are cursed with silly rules in your programming course, we offer you some word of encouragement to find a mentor and keep your mental sanity and programming habits.

Lately, we had a talk with one of our former interns who now happens to study informatics at university. He presented some code he had written for his programming course and we did a team code review. The review itself was a lot of fun and sparked quite a few discussions. At one point, we assessed the different implementation styles of a method, changing the rather complex single return code into an early return method. Our former intern (now student) listened to the solution and stated: “I am not allowed to do that.”

There was a sudden silence, as everyone tried to comprehend what that means.

The student explained: “my course instructor prefers the single return approach over the early return style”. Well, that’s one thing, we can handle different opinions. “And”, he continued, “he announced there will be a painful deduction of points if we don’t comply to this style.” When the course tried to discuss this point, the explanation given was: “the single return style is superior because the other style is frowned upon.”

We couldn’t believe it. But, as it turns out, there are many rules like the one above in this programming course. And nearly every rule is highly debatable if not plain wrong (in our perception).

There is no problem with the presentation of certain rules in a beginner’s programming course. Novices need clear and precise rules to learn, according to the Dreyfus Model of Skill Acquisition. The concept just doesn’t work for students that aren’t on the Novices level anymore. These students are explicitely forbidden to create more advanced solutions. They are discouraged to look into different programming styles because it will only harm their grades.

We can think of a possible explanation for this scenario: The assignments have to be evaluated by the course instructors. It takes a lot of hard work (and time) to evaluate hundreds of totally different solutions for the same problem. If the solutions are mostly similar in style and concepts, the evaluation is a lot easier and can be done without full understanding of the code.

This is a rather poor explanation. It says “don’t be too advanced in your field of study or you will be too troublesome to attend to”. This is essentially an anesthetization by decree. But the real problem arises when you realize that there won’t be any continuative programming courses. They will never teach you all the more advanced concepts or rectify the silly rules that should get you along as a beginner. After you’ve successfully mastered this course, the studying focusses on the more academic topics of our field. The next possibility to develop your programming skills in a professional setting is your first software development job.

We don’t have a practical solution to this problem. One obvious solution would be to have more instructors evaluate less assignment solutions in the same time, enabling them to dive deeper in the code and give better personalized feedback. This scenario lacks at least enough capable instructors. The reality shows that Novices level students (in the sense of the Dreyfus Model) are often taught by Advanced Beginner level instructors (called a “tutor”).

But we have a word of encouragement for all you students out there, feeling dumbed down by your instructors: It’s not your fault! Take your programming course rules with a (big) grain of salt and talk to other developers. If you don’t know anybody already in the industry, try to make contact with some fellow open source developer on the web. It’s really just the advice “Find a Mentor” from the book Apprenticeship Patterns (highly recommended for aspiring software developers) applied in real life.

Because if you don’t actively unlearn all these arbitrary rules or at least put them into perspective, you’ll start your professional developer career with the burden of some really antic code quirks.

Good luck and tell us your story, if you want.

Separate Master Data and Variable Data

If you come across an accumulation of data fields, you might want to split them into master data and variable data. This could at least help when dealing with storage issues.

In the design of data structures or objects, there are two different kinds of data, namely “Master Data” (german: Stammdaten) and “Variable Data” (german: Bewegungsdaten). The first kind, master data, are data fields that will change seldom over time and can sometimes be used to “identify” an object. The second kind are data fields that capture the current value of an object’s aspect, but are expected to change in the future. If you can categorize your data fields in this manner, think about separating them into different objects.

Let me make an actual example. An application we develop has a central instance (the “center”) that distributes situational data to several operation desks, powered by client applications, named the “clients”. Each client instance is registered in the center to enable the supervision and administration of clients. The data for each client is stored in a ClientInformation object that is mapped to a database relation. Let’s have a look at some of the data fields of ClientInformation:

  • int internalIdentifier – the database primary key for the record
  • String type – some type of the client application
  • String instanceName – the given readable denotation of the operation desk
  • String version – the currently installed version of the client application
  • Date connectionDate – the last time this client application established a connection
  • Date lastActionDate – the last time this client application issued an action command (“was active”)

We can start all kinds of (justified) discussion about primitive obsession, too much information at one place and so on, but for this blog entry, only the categorization in master data and variable data is of interest. My opinion on the example is that the first three data fields (internalIdentifier, type and instanceName) are definitely in the master data category. The last two data fields are clearly variable data, while the version field is something in between. My guts tell me to categorize the version as master data, because it won’t change on a daily schedule.

When separating the two categories of data, the ClientInformation object may turn into a reference holder object only. In this case, the ClientInformation holds two references, one to a new ClientMasterData object (holding internalIdentifier, type, instanceName and version) and another one to a new ClientVariableData object (holding connectionDate and lastActionDate).

A less radical modification would be to let the master data remain in the ClientInformation object and only extract the variable data into a new ClientConnectionData object. If a client connects, only the referenced ClientConnectionData object has to change.

If you separate your master data from the variable data, you can very easily concentrate on the variable data for performance optimizations. This is where the data changes will happen and a tuned storage strategy will pay off. The master data should be designed more carefully concerning the type information, so if we really start the discussion about primitive obsession, I would first tend to the master data fields and argue that the type shouldn’t be a String but an Enum and the version should be a more sophisticated Version type. This could be modelled even with a slow object/relational mapper because the data is only written/read once.

The next time you come across one of your data model objects that contain more than two data fields, have a look at their categorization in master and variable data. Perhaps you can see a good reason to split the object.

Depth-first programmers

Depth-first programmers are always busy creating horribly complicated solutions that are somehow off the mark. Here’s why and what to do against it.

Just as there are at least two fundamentally different approaches for searching, namely depth-first and breadth-first search, there are also different types of programmers. The depth-first programmer is a dangerous type, as he is prone to yak shaving and reinvention of the wheel.

The depth-first programmer

Let me try to define the term of a “depth-first programmer” by a little (true) story. A novice java programmer should make some changes to an existing code. To secure his work, he should and wanted to write unit tests in JUnit. He started the work and soon enough, first results could be seen. But when he started to write his tests, the progress notifications stopped. The programmer worked frantically for hours and then days to write what appeared to be some simple data-driven tests.

Finally, the novice java programmer reported success and showed his results. He wrote his tests and “had to extend JUnit a bit to do it right”. Wait, what? Well, in JUnit, the test methods cannot have parameters, but the programmer’s tests needed to be parametrized. So he replaced the part of JUnit that calls the test methods by reflection with an “improved” algorithm that could also inject parameters. His implementation relied on obscure data structures that provided the actual parameter values and only really worked for his needs. The whole mess was nearly intangible, a big bloat and needed most of the development time for the unit tests.

And it was totally unnecessary once you learn about “Parameterized” JUnit4 tests or build light-weight data drivers instead of changing the signature of the test method itself. But this programmer dove deep into JUnit to adjust the framework itself to his needs. When asked about it, he stated that “he needed to pass the parameters somehow”. That’s right, but he choose the most expensive way to do so.

He exhibited the general behaviour of a depth-first programmer: whenever you face a problem, take the first possible solution to a problem you can come up with and work on it without evaluation against other possibilities. And continue on the path without looking back, no matter how long it takes.

Stuck in activism

The problem with this approach should be common sense. The obvious option isn’t always the best or even a good one. You have to evaluate the different possible solutions by their advantages and drawbacks. A less obvious solution might be far better in every aspect but obviousness. Another problem with this approach is the absence of internal warning signs.

Getting stuck is an internal warning sign every programmer understands. You’ve worked your way in a certain direction and suddenly, you cannot advance further. Something blocks your anticipated way of solving the problem and you cannot think of an acceptable way past it. A depth-first programmer never gets stuck this way. No matter how expensive, he will pursue the first thing that brings him closer to the target. A depth-first programmer always churns out code at full speed. Most of it isn’t needed on second thought and can be plain harmful when left in the project. The depth-first programmer will always report progress even when he needs days for a task of minutes. He is stuck in activism.

Progress without guidance

This isn’t a rant about incompetent programmers. Every good programmer knows the situation when you suddenly realize that you’re shaving a yak when all you wanted to do is to add a feature to the code base. This is your self-guidance system regaining consciousness after a period of auto-piloting in depth-first mode. Every programmer behaves depth-first sometimes.

This can be explained with the Dreyfus Model of Skill Acquisition. On the first stage, called “Beginner”, you are simply not capable of proper self-evaluation. You cannot distinguish between good and not so good approaches beforehands or even afterwards. Your expertise in the narrow field of the problem at hand isn’t broad enough to recognize an error even when you are working on the error yourself for prolonged times.

In the Dreyfus Model, a beginner needs external guidance. Somebody with more experience has to point out errors for you and formulate alternatives as clearly and specific as possible. Without external guidance, a beginner will become a depth-first programmer. We’ve all been there.

 Be a guide

The real failure in the story above was done by me. Instead of interacting with the novice java programmer after a few hours when I thought he should be done by now, I let him “advance”. I could have avoided the resulting mess by providing guidance and a few alternate solutions for the immediate problem. I would give an overview of the problem’s context and some hints about the general direction this task should be solved.

Every depth-first programmer works in a suboptimal environment. The programmer tries his best, it’s really the environment that could do better.

So, the next time you see somebody working frantically on a problem that should be rather easy to solve, lend him a hand. Be gentle and empathic about his attempt and work with proposals, not with instructions. Perhaps you’ve spared yourself a mess like an unnecessarily extended JUnit library and the depth-first programmer the frustration when his hard work of several days is silently discarded.

A review of the year 2011 at Softwareschneiderei

This is a review of the year 2011 for the Softwareschneiderei, a software development company from Karlsruhe, Germany.

The current year 2011 is coming to an end. This is the traditional time to pause and reflect on what has happened. This blogpost tries to sum up our year in software development at Softwareschneiderei. It was an interesting, entertaining and successful year for us, that’s for sure.

The official parts

Our developer blog was alive throughout the hardest times of the year, when everyone was under full project load. Every week, one of our developer shares a little posting with the world. The blog is still managed by token only. Looking at the visitor statistics, we fully appreciate your attention. The first blog post of this year looked at the remainder of a failed project. “A tale of scrap metal code” was a detailed vivisection in three parts. Over the course of the year, we wrote about bogus error messages, Groovy, Grails, GORM and some confessions about coding style and multithreading. If you have the time, spend a few minutes to browse our blog post archive for this year.

Our “official” company blog, written in german language, had no activity this year. We can certainly do better than this and take it on the list for 2012.

The company homepage, written in german language, had continuous updates and extensions this year. We coupled the “Open Source Love Day” (OSLD) with our “Homepage Comittee”, when every employee has to improve the homepage in some aspects and present the change to the “comittee”. Unfortunately, this somehow lead to fewer OSLDs this year.

The ongoing Dev Brunch sessions thinned out a bit in the second half of the year. This was a concession to the ever-growing workload. We strive to establish a tighter schedule with accompanying blog posts in the next year.

The internal parts

We were under heavy development load this year. This isn’t a bad thing, but impacts the internal communication and team building process. We tried to cope a bit by restructuring the Open Source Love Day to a “Team Day”, when the whole team meets and works on various internal or hobby projects. Some of these days were spend on Code Camps and other training events. This means less love for the open source community, but crucial together time for us.

We picked up several “new” programming languages this year. You can tell by the blog posts that we worked with Python, Ruby, Flex/ActionScript and even VisualBasic on real projects. The VisualBasic experience was a little epiphany that it’s really the developer and not the language that leads to shitty software.

One method of continuous improvement is our “creativity budget”. Basically, it’s money every developer can spend to improve his workplace. This budget wasn’t used at all this year, as the workplaces seem to be optimal. This cannot be true, so we bought brand new computers or a big RAM upgrade for everyone. And every computer has a big enough SSD now. We took our own advice seriously and invested in our productivity.

Our developer crew grew again this year. We are beginning to think about the remaining space in the new office again. But as usual, we grow slowly and deliberately. There’s nothing worse than a team of strangers.

Conclusion

The year 2011 was great! We’re looking forward to the year 2012, with our motto of christmas 2011: “cheery and spry” (the original motto is in german language “froh und munter”, I hope the translation caught the original spirit).

Have a great turn of the year, everyone. We’d love to see you again next year.

The Story of a Multithreading Sin

The story of a bug that was caused by a common multithreading pitfall, the dreaded liquid lock.

In my last blog entry, I wrote about multithreading pitfalls (in Java), and ironically, this was the week when we got a strange bug report from one of our customers. This blog entry tells the story of the bug and adds another multithreading pitfall to the five I’ve already listed in my blog entry “When it comes to multithreading, better be safe than sorry”.

The premise

We developed a software that runs on several geographically distant independent “stations” that collect a multitude of environmental measurement data. This data is preprocessed and stuffed into data packages, which are periodically transferred to a control center. The software of this control center, also developed by us, receives the data packages, stores them on disk and in a huge database and extracts the overall state of the measurement network from raw data. If you describe the main task of the network on this level, it sounds nearly trivial. But the real functionality requirements are manifold and the project grew large.

We kept the whole system as modular as necessary to maintain an overall grasp of what is going on where in the system and installed a sufficient automatic test coverage for the most important parts. The system is still under active development, but the main parts of the network are in production usage without real changes for years now.

The symptoms

This might explain that we were very surprised when our customer told us that the control center had lost some data packages. Very soon, it turned out that the control center would randomly enter a state of “denial”. In this state, it would still accept data packages from the stations and even acknowledge their arrival (so the stations wouldn’t retry the transmission), but only write parts of the package or nothing at all to the disk and database. When the control center entered this state, it would never recover from it. But when we restarted the software manually, everything would run perfectly fine for several days and then revert back into denial without apparent trigger.

We monitored the control center with every means on our disposal, but its memory consumption, CPU footprint and threading behaviour was without noticeable problem even when the instance was in its degraded state. There was no exception or uncommon entry logged in the logfiles. As the symptom happened randomly, without external cause and with no chance of reversal once it happened, we soon suspected some kind of threading issue.

The bug

The problem with a threading issue is that you can’t just reproduce the bug with an unit or system test. We performed several code reviews until we finally had a trace. When a data package arrives, a global data processing lock is acquired (so that no two data packages can be processed in parallel) and the content of the package is inspected. This might trigger several network status changes. These change events are propagated through the system with classic observer/listener structures, using synchronous calls (normal delegation). The overall status of the network is translated in a human readable status message and again forwarded to a group of status message listeners. This is a synchronous call again. One of the status message listeners was the software driver for a LED ticker display. This module was a recent addition to the control center’s hardware outfit and used to display the status message prominently to the operators. Inside this LED software driver, some bytes are written to a socket stream and then the driver awaits an answer of the hardware device. To avoid the situation that two messages are sent to the device at the same time, a lock is acquired just before the message is sent. This code attracted our attention. Lets have a look at it:

private Message lastMessage = new Message();

public void show(Message message) {
    synchronized (this.lastMessage) {
        writeCommandAndWaitForResponse(Command.SHOW_TEXT, message.asBytes());
        this.lastMessage = message;
    }
}

The main problem here is the object the lock is acquired upon: the reference of lastMessage is mutable! We call this a liquid lock, because the lock isn’t as solid as it should be. It’s one of the more hideous multithreading pitfalls as it looks like everything’s fine at first glance. But this lock doesn’t have a complete “locking” effect because each caller may acquire the lock of a different instance. And a lock with a flawed locking behaviour is guaranteed to fail (in production). The liquid lock is like the bigger brother of the local lock. It isn’t local, but its mutability cause the same problems.

The bug finally turned out to be caused by the liquid lock in the LED display driver that got notified of system message changes when a data package arrived. But only if multiple messages were sent at once to the device, discarding some of the necessary answers in this circumstance or if the connection to the LED hardware would fail in the midst of a transmission, the system would not return from the write attempt. If one thread wouldn’t return to the data package processor, the global data processing lock would not be freed (read the start of this chapter again, this is the most important lock in the system!). And while the data processing lock was still held, all other data packages would be received, but piling up to obtain the lock. But the lock would never be returned from the thread waiting on an answer from a hardware device that had no intention to send another answer. This was when the control center appeared to be healthy but didn’t process any data packages anymore.

The conclusion

If you want to avoid the category of liquid lock multithreading bugs, make sure that all your lock instance references are immutable. Being final is an important property of lock instance references. Avoid to retrieve your locks from notoriously muteable data structures like collections or arrays. The best thing you can do to avoid liquid locks is to “freeze” all your lock instances.

Another insight from this story is that software modules have to be separated threadwise, too. It was a major design flaw to let the data processing thread, while holding the main processing lock, descend down into the deep ends of the LED driver, eventually getting stuck there for infinity. Some simple mechanisms like asynchronous listener notification or producer/consumer queues for pending transmission requests would have helped to confine the effects of the liquid lock bug inside the LED module. Without proper thread separation, it took down the whole software instance.

When it comes to multithreading, better be safe than sorry

Writing multithreaded applications in Java is hard. Here are five problems and how to avoid them without much effort (mostly).

Recently, I attended a code review of the core parts of a web application, written in Java. The application is used by a large customer base and occassionally, there are error reports and exceptions in the log files. Some of these exceptions are the dreaded ConcurrentModificationExceptions, indicating conflicting read/write access on an unsynchronized collection data structure. In the code review, we found several threading flaws, but not after an exhaustive reading of the whole module. Here, I want to present the flaws and give some advice on how to avoid them:

The public lock

In some parts of the code, methods were defined as synchronized through the method declaration keyword:

public synchronized String getLastReservation() { [...]

While there is nothing wrong with this approach in itself, it can be highly dangerous in combination with synchronized blocks. The code above effectively wraps a synchronized block using the object instance (this) as a lock. No information of an object is more publicly visible as the object reference (this), so you have to check all direct or indirect clients of this object if they synchronize on this instance, too. If they do, you have chained two code blocks together, probably without proper mentioning of this fact. The least harmful defect will be performance losses because your code isn’t locked as fine grained as it could be.

The easiest way to avoid these situations it to always hide the locks. Try not to share one object’s locks with other objects. If you choose publicly accessible locks, you can never be sure about that.

The subtle lock change

In one class, there were both instance and class (static) methods, using the synchronized keyword:

public synchronized String getOrderNumberOf(String customerID) { [...]
public  synchronized static int getTotalPendingOrders() { [...]

And while they were both accessing the same collection data structure (a static hashmap), they were using different locks. The lock of the instance method is the instance itself, while the lock of the static method is the class object of the type. This is very dangerous, as it can be easily missed when writing or altering the code.

The best way to prevent this problem it to avoid the synchronized modifier for methods completely. State your locks explicitely, all the time.

Partial locking

In a few classes, collection datatypes like lists were indeed synchronized by internal synchronized-blocks in the methods, using the private collection instance as lock. The synchronized blocks were applied to the altering methods like putX(), removeX() and getX(). But the toString() method, building a comma-separated list of the textual list entries, wasn’t synchronized to the list. The method contained the following code:

public String toString() {
    StringBuilder result = new StringBuilder();
    for (String entry : this.list) {
        result.append(entry);
        result.append(",");
    }
    [...]
    return result.toString();
}

I’ve left out some details and special cases, as they aren’t revelant here. The problem with the foreach loop is that an anonymous Iterator over the list is used and it will relentlessly monitor the list for any changes and throw a ConcurrentModificationException as soon as one of the properly synchronized sections changes it. The toString() method was used to store the list to a session dependent data storage. Every once in a while, the foreach loop threw an exception and failed to properly persist the list data, resulting in data loss.

The most straight-forward solution to this problem might be to add the missing synchronization block in the toString() method. If you don’t want to block the user session while writing to disk, you might traverse the list without an Iterator (and be careful with your assumptions about valid indices) or work on a copy of the list, given that an in-memory copy of the list would be cheap. In an ACID system scenario, you should probably choose to complete your synchronized block guards.

Locking loophole

Another problem was a collection that was synchronized internally, but could be accessed through a getter method. No client could safely modify or traverse the collection, because they had the collection, but not the lock object (that happened to be the collection, too, but who can really be sure about that in the future?). It would be ridiculous to also provide a getter for the lock object (always hide your locks, remember?), the better solution is to refactor the client code to a “tell, don’t ask” style.

To prevent a scenario when a client can access a data structure but not its lock, you shouldn’t be able to gain access to the data structure, but pass “command objects” to the data structure. This is a perfect use case for closures. Effectively, you’ll end up with something like Function or Operation instances that are applied to every element of the collection within a synchronized block and perform your functionality on them. Have a look at op4j for inspirational syntax.

Local locking

This was the worst of all problems and the final reason for this blog entry: In some methods, the lock objects were local variables. In summary, these methods looked like this:

public String getData() {
    Object lock = new Object();
    synchronized (lock) {
        [...]
    }
}

Of course, it wasn’t that obvious. The lock objects were propagated to other methods, stored in datastructures, removed from them, etc. But in the end, each caller of the method got his own lock and could henceforth wreck havoc in code that appeared very well synchronized on first look. The error in its clarity is too stupid to be widespread. The problem was the obfuscation around it. It took us some time to really understand what is going on and where all that lock objects really come from.

My final advice is: If you have to deal with multithreading, don’t outsmart yourself and the next fellow programmer by building complex code structures or implicit relationships. Be as concise and explicit as you can be. Less clutter is more when dealing with threads. The core problem is the all-or-none law of thread synchronization: Either you’ve got it all right or you’ve got it all wrong – you just don’t know yet.

Hide your locks, name your locks explicitely, reduce the scope of necessary locking so that you can survey it easily, never hand out your locked data, and, most important, remove all clutter around your locking structures. This might make the difference between “just works” and endless ominous bug reports.