Author: daniel.lindner

Summary of the Schneide Dev Brunch at 2012-07-15

If you couldn’t attend the Schneide Dev Brunch in July 2012, here are the main topics we discussed neatly summarized.

Two weeks ago, we held another Schneide Dev Brunch. The Dev Brunch is a regular brunch on a sunday, only that all attendees want to talk about software development and various other topics. If you bring a software-related topic along with your food, everyone has something to share. The brunch was so well attended that we had trouble to all find a chair and fit at the table. There were quite some new participants, so the introductory round was necessary again. Let’s have a look at the main topics we discussed:

Choosing Google Web Toolkit

If you start the development of a new web application today, there are many frameworks to call to aid. Most of them will not lend a hand, but mostly stand in the way. In a short presentation, we learnt about the arguments in favor and against the use of the Google Web Toolkit for the development of a highly customizable web application. The two most important aspects were that GWT enables desktop-like “real” (as opposed to “web”) development but still provides enough hooks to embrace the web-only developers.

Spock Framework

The Spock testing framework tries to bring natural and expressive syntax back to testing. It mixes the best of most current testing and specification frameworks together in a groovy-based domain specific language. The first contact with Spock of one of our attendees was very pleasant. The framework provides opiniated default tools for most modern testing aspects (e.g. mocking), but is extremely integrative with all current testing libraries. The take-away of this topic was: Try Spock for your next adventures in testing.

Schneide job offer

We from the Softwareschneiderei host the Dev Brunch for nearly six years now. In all these years, we grew slowly without the need to announce open job offers. Now is the time where even we have to insert a little bit of advertising into the brunch: we are hiring. Enough said.

SWT UI-based tests

The Standard Widget Toolkit is the graphical foundation of the Eclipse platform. It’s a bit dated (like most Java-based UI toolkits) and doesn’t really embrace automatic UI-based tests. There is SWTBot, but it doesn’t provide the power of a tool like FEST-Swing. We discussed the situation, but couldn’t offer much help.

Decorator pattern

Another question for discussion was the usage of the classic decorator design pattern in a rather twisted use case. Without going into much detail, the best option would have been the use of Mixins, but the environment (Java) doesn’t provide them. It was an interesting discussion with lots of different solution attempts. We didn’t find the definitive answer, but there was some good inspiration in the train of thoughts.

The programming language Go

One guest offered us a quick overview over the new programming language Go, developed by Google. To sum up a few aspects that were mentioned, Go is compiled to native code, doesn’t offer type inheritance but compile-time interface binding (if it matches, it binds) and so-called goroutines. The latter are slightly improved coroutines. The communication between objects is mostly done with channels, a very flexible event notification system. The feature with the most raised eyebrows was the visibility modifier: capitalization. If your name begins with a capital letter, it is exported. You can quickly learn the basics of the language with the web-based “Tour of Go”.

Summary of Java Forum Stuttgart

One attendee summed up his impressions and experiences with this year’s Java Forum Stuttgart, a local Java-based conference. The day was worthwile and informative, but some basic pieces went missing. For example, you weren’t provided with a notepad and a pen and had to go on a hunt at the exhibitor booths. The single extraordinary talk that you’ll remember for years didn’t happen, either. Most visited talks were solid, but not outstanding. Most noteworthy tools this year were Gerrit and Sonar.

Data loses its location

A final aspect for open discussion was the fact that stored data loses its location. With all modern web- or cloud-based services, the notion of a “storage medium” begins to lose meaning. And with the fast mobile internet access, you won’t have to reside at a specific location to access all your data. For the next generations of computer users (read: kids), data will behave like the notorious aether: always there, never affixable.

Epilogue

As usual, the Dev Brunch contained a lot more chatter and talk than listed here. The high number of attendees makes for an unique experience every time. We are looking forward to the next Dev Brunch at the Softwareschneiderei. And as always, we are open for guests and future regulars. Just drop us a notice and we’ll invite you over next time.

A short story about priorities

When you mix your own priorities with the ones of your customers, embarrassing things might happen.

Before I can tell you the story, I have to add this disclaimer: The story is true, but it wasn’t us. To protect the identities, I’ve changed nearly all the details, hopefully without watering down the essence of the story.

The assignment

A few years ago, when developing software for mobile devices wasn’t as ordinary as it is today, a freelance developer got a call from a loyal customer. He should develop a little web application that could also be used on a smartphone. The functionality of the application was minimal: Show a neat image containing two buttons to download a song teaser in two different file formats, probably MP3 and WMA. That’s all. The whole thing was a marketing app that should be used a few weeks or months and then be mothballed. Being a marketing app, the payment was very decent.

The development

The freelancer was excited to say the least. This was his chance to enter the emerging field of mobile device software development. And because it should be a web application, he would try out all these new fancy mobile web frameworks and settle for the best. The image of the website should rotate and adjust to the device screen if the user tilted his phone. And because phone screens weren’t that well equipped with pixels as today and mobile internet was expensive, the image should be optimized for the best possible tradeoff between details and file size.

Some mobile web frameworks provided access to the device type, so there were all kinds of javascript magic tricks to be potentially applied. Well, they weren’t applied, but very well could have been. The freelancer prototyped most of them, out of curiosity.

The web application itself was completed in one session once the song files and the master image were provided by the customer. The freelancer installed everything on a staging web server and communicated back success. The customer was pleased by the quick reaction time and appointed a meeting to inspect the results.

The presentation

The customer gave the freelancer a notebook attached to a beamer to present the application. The notebook had a screen resolution many times the envisioned target platform, so the intro image was very pixellated and far from appealing. The freelancer used his smartphone to present the image on a smaller screen, but the customer only shook his head: “You should develop a web application for both phones and PCs. Most of our customers will use their PC to visit the site.” The freelancer apologized and promised to add another browser switch to present a full resolution image to PC users.

Now, the freelancer continued to present the web application on his phone. He invited the customer to tilt the device and was satisfied when the web site adjusted perfectly. The customer was pleased, too. Then, he tapped on one of the buttons. Nothing happened. The whole application consisted only of two buttons and one wasn’t working. The freelancer frantically tried to figure out what had gone wrong, when the customer tapped on the other button. No reaction from that one, too. “But I’ve uploaded both song files to the right location with the right access rights.”, the freelancer just said to himself when it dawned him: He forgot to insert the link tags in the HTML file. The buttons were just images. He never actually clicked on one of the buttons during development.

The moral of the story

Recapitulating, the freelancer was asked to develop a web application with an image and two download buttons, but he managed to cripple the image for the larger part of the anticipated users and never provide any download functionality at all. The customer’s requirements somehow got lost along the way.

This isn’t so much a story about a confused freelancer or improper requirement analysis. It’s a story about priorities. The customer expressed his priorities through the requirements. The freelancer superimposed his own priorities very early in the process (without telling anybody) and never returned to the original set until the presentation. And while it is granted to have a secondary agenda as a service provider, it should never interfere with the primary agenda – the one set by the customer.

Don’t fool yourself into thinking that this could never happen to you. It’s not always as obvious as in this story. Some, if not most of your customer’s priorities are unintentionally (or even intentionally) kept secret. They can only be traced during live exercises, which is why early prototyping and using the prototype in real scenarios is a good way to reveal them.

Summary of the Schneide Dev Brunch at 2012-05-27

If you couldn’t attend the Schneide Dev Brunch in May 2012, here are the main topics we discussed neatly summarized.

Yesterday, we held another Schneide Dev Brunch on our roofgarden. The Dev Brunch is a regular brunch on a sunday, only that all attendees want to talk about software development and various other topics. If you bring a software-related topic along with your food, everyone has something to share.

We had to do another introductory round because there were new participants with new and very interesting topics. This brunch was very well attended and rich in information. Let’s have a look at the main topics we discussed:

Agile wording (especially SCRUM)

This was just a quick overview over the common agile vocabulary and what ordinary people associate with them. A few examples are “scrum“, “sprint” and “master”. We agreed that some terms are flawed without deeper knowledge about the context in agile.

Book: “Please Understand Me”

if you are interested in the Myers-Briggs classification of personality types (keywords: are you INTJ, ESTP or INFP?), this is the book to go. It uses a variation of the personality test to classify and explain yourself, your motives and personal traits. And if you happen to know about the personality type of somebody else, it might open your eyes to the miscommunication that will likely occur sooner or later. Just don’t go overboard with it, it’s just a model about the most apparent personality characteristics. The german translation of the book is called “Versteh mich bitte” and has some flaws with typing and layouting errors. If you can overlook them, it might be the missing piece of insight (or empathy) you need to get through to somebody you know.

TV series: “Dollhouse”

As most of us are science fiction buffs and hold a special place in our heart for the series “Firefly”, the TV series “Dollhouse” by Joss Whedon should be a no-brainer to be interested in. This time, it lasted two seasons and brings up numerous important questions about programmability every software developer should have a personal answer for. Just a recommendation if you want to adopt another series with limited episode count.

Wolfpack Programming

A new concept of collaborative programming is “wolfpack programming” (refer to pages 21-26). It depends on a shared (web-based) editor that several developers use at once to develop code for the same tasks. The idea is that the team organizes itself like a pack of wolves hunting deer. Some alpha wolves lead groups of developers to a specific task and the hunt begins. Some wolves/developers are running/programming while the others supervise the situation and get involved when convenient. The whole code is “huntable”, so it sounds like a very chaotic experience. There are some tools and reports of experiments with wolfpack programming in Smalltalk. An interesting idea and maybe the next step beyond pair programming. Some more information about the editor can be found on their homepage and in this paper.

Book: “Durchstarten mit Scala”

Sorry for the german title, but the book in this review is a german introductory book about Scala. It’s not very big (around 200 pages) but covers a lot of topics in short, with a list of links and reading recommendations for deeper coverage. If you are a german developer and used to a modern object-oriented language, this book will keep its promise to kickstart you with Scala. Everything can be read and understood easily, with only a few topics that are more challenging than there are pages for them in the book. The topics range from build to test and other additional frameworks and tools, not just core Scala. This book got a recommendation for being concise, profound and understandable (as long as you can understand german).

Free Worktime Rule

This was a short report about employers that pay their developers a fixed salary, but don’t define the workload that should happen in return. Neither the work time nor the work content is specified or bounded. While this sounds great in the first place (two hours of work a week with full pay, anybody?), we came to the conclusion that peer pressure and intrinsic motivation will likely create a dangerous environment for eager developers. Most of us developers really want to work and need boundaries to not burn out in a short time. But an interesting thought nevertheless.

Experimental Eclipse Plugin: “Code_Readability”

This was the highlight of the Dev Brunch. One attendee presented his (early stage) plugin for Eclipse to reformat source code in a naturally readable manner. The effect is intriguing and very promising. We voted vehemently for early publication of the source code on github (or whatever hosting platform seems suitable). If the plugin is available, we will provide you with a link. The plugin has a tradition in the “Three refactorings to grace” article of the last Dev Brunch.

Light Table IDE

A short description of the new IDE concept named “Light Table”. While the idea itself isn’t new at all, the implementation is very inspirational. In short, Lighttable lets you program code and evaluates it on the fly, creating a full feedback loop in milliseconds. The effects on your programming habits are… well, see and try it for yourself, it’s definitely worth a look.

Inventing on Principles

Light Table and other cool projects are closely linked to Bret Victor, the speaker in the mind-blowing talk “Inventing on Principles”. While the talk is nearly an hour of playtime, you won’t regret listening. The first half of the talk is devoted to several demo projects Bret made to illustrate his way of solving problems and building things. They are worth a talk alone. But in the second half of the talk, Bret explains the philosophy behind his motivation and approach. He provides several examples of people who had a mission and kept implementing it. This is very valuable and inspiring stuff, it kept most of us on the edge of our seats in awe. Don’t miss this talk!

Albatros book page reminder (and Leselotte)

If you didn’t upgrade your reading experience to e-book readers yet, you might want to look at these little feature upgrades for conventional books. The Albatros bookmark is a page remembering indexer that updates itself without your intervention. We could test it on a book and it works. You might want to consider it especially for your travelling literature. This brought us to another feature that classic dead wood books are lacking: the self-sustained positioning. And there’s a solution, too: The “Leselotte” is a german implementation of the bean bag concept for a flexible book stand. It got a recommendation by an attendee, too.

Bullshit-Meter

If you ever wondered what you just read: It might have been bullshit. To test a text on its content of empty phrases, filler and hot air, you can use the blabla-meter for german or english text. Just don’t make the mistake to examine the last apidoc comments you hopefully have written. It might crush your already little motivation to write another one.

Review on Soplets

In one of the last talks on the Java User Group Karlsruhe, there was a presentation of “Soplets”, a new concept to program in Java. One of our attendees summarized the talk and the concept for us. You might want to check out Soplets on your own, but we weren’t convinced of the approach. There are many practical problems with the solution that aren’t addressed yet.

Review on TDD code camp

One of our attendees lead a code camp with students, targeting Test Driven Development as the basic ruleset for programming. The camp rules closely resembled the rules of Code Retreats by Corey Haines and had Conway’s Game of Life as the programming task, too. With only rudimentary knowledge about TDD and Test First, the students only needed four iterations to come up with really surprising and successful approaches. It was a great experience, but showed clearly how traditional approaches like “structured object-oriented analysis” stands in the way of TDD. Example: Before any test was written to help guide the way, most students decided on the complete type structure of the software and didn’t aberrate from this decision even when the tests told them to.

Report of Grails meetup

Earlier last week, the first informal Grails User Group Karlsruhe meeting was held. It started on a hot late evening some distance out of town in a nice restaurant. The founding members got to know each other and exchanged basic information about their settings. The next meeting is planned with presentations. We are looking forward to what this promising user group will become.

Epilogue

This Dev Brunch was a lot of fun and new information and inspiration. As always, it had a lot more content than listed here, this summary is just a best-of. We are looking forward to the next Dev Brunch at the Softwareschneiderei. And as always, we are open for guests and future regulars. Just drop us a notice and we’ll invite you over next time.

Summary of the Schneide Dev Brunch at 2012-03-25

If you couldn’t attend the Schneide Dev Brunch in March 2012, here are the main topics we discussed for you to review.

This summary is a bit late and my only excuse it that the recent weeks were packed with action. But the good news is: The Schneide Dev Brunch is still alive and gaining traction with an impressive number of participants for the most recent event. The Schneide Dev Brunch is a regular brunch in that you gather together to have a late breakfast or early dinner on a sunday, only that all attendees want to talk about software development (and various other topics). If you bring a software-related topic along with your food, everyone has something to share. We were able to sit in the sun on our roofgarden and enjoy the first warm spring weekend.

We had to do introductory rounds because there were quite some new participants this time. And they brought good topics and insights with them. Let’s have a look at the topics we discussed:

Checker Framework

This isn’t your regular java framework, meant to reside alongside all the other jar files in your dependency folder. The Checker framework enhances java’s type system with “pluggable types”. You have to integrate it in your runtime, your compiler and your IDE to gain best results, but after that you’re nothing less than a superhero among regulars. Imagine pluggable types as additional layers to your class hierarchy, but in the z-axis. You’ll have multiple layers of type hierachies and can include them into your code to aid your programming tasks. A typical use case is the compiler-based null checking ability, while something like Perl’s taint mode is just around the corner.

But, as our speaker pointed out, after a while the rough edges of the framework will show up. It still is somewhat academic and lacks integration sometimes. It’s a great help until it eventually becomes a burden.

Hearing about the Checker framework left us excited to try it sometimes. At least, it’s impressive to see what you can do with a little tweaking at the compiler level.

Getting Stuck

A blog entry by Jeff Wofford inspired one of us to talk about the notion of “being stuck” in software development. Jeff Wofford himself wrote a sequel to the blog entry, differentiating four kinds of stuck. We could relate to the concept and have seen it in the wild before. The notion of “yak shaving” entered the discussion soon. In summary, we discussed the different types of being stuck and getting stuck and what we think about it. While there was no definite result, everyone could take away some insight from the debate.

Zen to Done

One topic was a review of the Zen to Done book on self-organization and productivity improvement. The methodology can be compared to “Getting Things Done“, but is easier to begin with. It defines a bunch of positive habits to try and establish in your everyday life. Once you’ve tried them all, you probably know what works best for you and what just doesn’t resonate at all. On a conceptional level, you can compare Zen to Done to the Clean Code Developer, both implementing the approach of “little steps” and continuous improvement. Very interesting and readily available for your own surveying. There even exists a german translation of the book.

Clean Code Developer mousepads

Speaking of the Clean Code Developer. We at the Softwareschneiderei just published our implementation of mousepads for the Clean Code Developer on our blog. During the Dev Brunch, we reviewed the mousepads and recognized the need for an english version. Stay tuned for them!

Book: Making software

The book “Making software” is a collection of essays from experienced developers, managers and scientists describing the habits, beliefs and fallacies of modern software development. Typical for a book from many different authors is the wide range of topics and different quality levels in terms of content, style and originality. The book gets a recommendation because there should be some interesting reads for everyone inside. One essay was particularly interesting for the reviewer: “How effective is Test-Driven Development?” by Burak Turhan and others. The article treats TDD like a medicine in a clinical trial, trying to determine the primary effects, the most effective dosage and the unwanted side effects. Great fun for every open-minded developer and the origin of a little joke: If there was a pill you could take to improve your testing, would a placebo pill work, too?

Book: Continuous Delivery

This book is the starting point of this year’s hype: “Continuous Delivery” by Jez Humble and others. Does it live up to the hype? In the opinion of our reviewer: yes, mostly. It’s a solid description of all the practices and techniques that followed continuous integration. The Clean Code Developer listed them as “Continuous Integration II” until the book appeared and gave them a name. The book is a highly recommened read for the next years. Hopefully, the practices become state-of-the-art for most projects in the near future, just like it went with CI. The book has a lot of content but doesn’t shy away from repetition, too. You should read it in one piece, because later chapters tend to refer to earlier content quite often.

Three refactorings to grace

The last topic was the beta version of an article about the difference that three easy refactorings can make on test code. The article answered the statement of a participant that he doesn’t follow the DRY principle in test code in a way. It is only available in a german version right now, but will probably be published on the blog anytime soon in a proper english translation.

Epilogue

This Dev Brunch was a lot of fun and had a lot more content than listed here. Some of us even got sunburnt by the first real sunny weather this year. We are looking forward to the next Dev Brunch at the Softwareschneiderei. And as always, we are open for guests and future regulars. Just drop us a notice and we’ll invite you over next time.

Clean Code Developer at your fingertips

You’ve probably already heard about the Clean Code Developer initiative. We’re donating a full spectrum of mousepad designs for your educational support.

We are participants in the Clean Code Developer (CCD) movement. This initiative provides a way to perpetually learn, train, reflect and act on the most important topics of today’s software development by formulating a value system and a learning path. The learning path is subdivided in different grades, associated with colors. Every Clean Code Developer progresses continually through the grades, focussing on the principles and practices of the current grade.

If you want a tongue-in-cheek explanation of what the Clean Code Developer is in one sentence: It’s a sight-seeing tour to the most prominent topics every professional software developer should know. But other than your usual tourist rip-off, you can just stay seated and enjoy another round without ever paying anything except attention.

Visualize it

An important aspect of learning and deliberate practice is proper visualization. We invest a lot of work at our workplace, our software and the interaction with our customers to make things visible. When we reflected on our Clean Code Developer practice, we knew that it lacked visualization.

The proposed equipment for a Clean Code Developer is a desktop background picture, a mousepad with an image of all grades at once and some rubber wristbands in the colors of the grades. The wristbands serve as a reminder and a self-assessment tool. The desktop background picture is nice, but only visible if we don’t perform actual work. This let us concentrate on the mousepad.

Duplicate if necessary

The mousepad is the most prominent “advertising” space on the typical work desktop. We want to advertise the content of our current Clean Code Developer grade to ourself. The combination of these two thoughts is not one mousepad, but one for every grade. Imagine six mousepads in the colors of the grades, displaying your currently most important topics right under your fingertips.

We liked the idea so much that we worked on it. The result is a collection of mousepads for every Clean Code Developer to enjoy.

Iterative design

It took us several full cycles of planning, design, layout and proof-reading to have the first version of mousepads produced. It took only a few hours of real-world testing to start the second iteration to further improve the design. Right now, we are on the third iteration. The first iteration had the five colored CCD grades printed on real mousepads. The second iteration added the mousepad for the white grade and a little stand-up display for the initial black grade. The third iteration incorporated the official Clean Code Developer logo, the website URL and improved some details.

Here are some promotional photos of the five first-iteration mousepads:

This slideshow requires JavaScript.

As you can see, we chose to print ultra-slim mousepads to test if it’s feasible to use them stacked all at once (it isn’t, your mileage may vary) or use them even if you aren’t used to mousepads at all (it depends, really). You might want to print the images onto the mousepad you prefer best.

Do it yourself

Yes, you’ve read it right. We are donating the mousepad images back to the community. You can download everything right here:

All documents are bare of any company logo or other advertising and free for your constructive usage. There is only one catch really: the documents are in german language. This might not be apparent at first because we really like the original english technical terms, but some content might need translation for non-german speakers. If you are interested to produce an all-english version, drop us a line.

Acknowledgements

These mousepads wouldn’t exist without the help and inspiration of many co-workers. First of all, the founders of the Clean Code Developer movement, Ralf Westphal and Stefan Lieser, provided all the content of the mousepads. Without their groundbreaking work, we probably wouldn’t have thought of this. The design and production is owed to Hannegret Lindner from the Hannafaktur, a small graphic design agency. We admire her endurance with our iterative approach. And finally, the initial inspiration sparked in a creative discussion with Eric Wolf and Benjamin Trautwein from ABAS Software AG.

It’s your turn now

We are very curious about your story, photo or action still with the mousepads (or the little stand-up display). You can also just share your thoughts about the whole idea or submit an improvement. We’d love to hear from you.

Take your programming course with a grain of salt, please

If you are cursed with silly rules in your programming course, we offer you some word of encouragement to find a mentor and keep your mental sanity and programming habits.

Lately, we had a talk with one of our former interns who now happens to study informatics at university. He presented some code he had written for his programming course and we did a team code review. The review itself was a lot of fun and sparked quite a few discussions. At one point, we assessed the different implementation styles of a method, changing the rather complex single return code into an early return method. Our former intern (now student) listened to the solution and stated: “I am not allowed to do that.”

There was a sudden silence, as everyone tried to comprehend what that means.

The student explained: “my course instructor prefers the single return approach over the early return style”. Well, that’s one thing, we can handle different opinions. “And”, he continued, “he announced there will be a painful deduction of points if we don’t comply to this style.” When the course tried to discuss this point, the explanation given was: “the single return style is superior because the other style is frowned upon.”

We couldn’t believe it. But, as it turns out, there are many rules like the one above in this programming course. And nearly every rule is highly debatable if not plain wrong (in our perception).

There is no problem with the presentation of certain rules in a beginner’s programming course. Novices need clear and precise rules to learn, according to the Dreyfus Model of Skill Acquisition. The concept just doesn’t work for students that aren’t on the Novices level anymore. These students are explicitely forbidden to create more advanced solutions. They are discouraged to look into different programming styles because it will only harm their grades.

We can think of a possible explanation for this scenario: The assignments have to be evaluated by the course instructors. It takes a lot of hard work (and time) to evaluate hundreds of totally different solutions for the same problem. If the solutions are mostly similar in style and concepts, the evaluation is a lot easier and can be done without full understanding of the code.

This is a rather poor explanation. It says “don’t be too advanced in your field of study or you will be too troublesome to attend to”. This is essentially an anesthetization by decree. But the real problem arises when you realize that there won’t be any continuative programming courses. They will never teach you all the more advanced concepts or rectify the silly rules that should get you along as a beginner. After you’ve successfully mastered this course, the studying focusses on the more academic topics of our field. The next possibility to develop your programming skills in a professional setting is your first software development job.

We don’t have a practical solution to this problem. One obvious solution would be to have more instructors evaluate less assignment solutions in the same time, enabling them to dive deeper in the code and give better personalized feedback. This scenario lacks at least enough capable instructors. The reality shows that Novices level students (in the sense of the Dreyfus Model) are often taught by Advanced Beginner level instructors (called a “tutor”).

But we have a word of encouragement for all you students out there, feeling dumbed down by your instructors: It’s not your fault! Take your programming course rules with a (big) grain of salt and talk to other developers. If you don’t know anybody already in the industry, try to make contact with some fellow open source developer on the web. It’s really just the advice “Find a Mentor” from the book Apprenticeship Patterns (highly recommended for aspiring software developers) applied in real life.

Because if you don’t actively unlearn all these arbitrary rules or at least put them into perspective, you’ll start your professional developer career with the burden of some really antic code quirks.

Good luck and tell us your story, if you want.

Separate Master Data and Variable Data

If you come across an accumulation of data fields, you might want to split them into master data and variable data. This could at least help when dealing with storage issues.

In the design of data structures or objects, there are two different kinds of data, namely “Master Data” (german: Stammdaten) and “Variable Data” (german: Bewegungsdaten). The first kind, master data, are data fields that will change seldom over time and can sometimes be used to “identify” an object. The second kind are data fields that capture the current value of an object’s aspect, but are expected to change in the future. If you can categorize your data fields in this manner, think about separating them into different objects.

Let me make an actual example. An application we develop has a central instance (the “center”) that distributes situational data to several operation desks, powered by client applications, named the “clients”. Each client instance is registered in the center to enable the supervision and administration of clients. The data for each client is stored in a ClientInformation object that is mapped to a database relation. Let’s have a look at some of the data fields of ClientInformation:

  • int internalIdentifier – the database primary key for the record
  • String type – some type of the client application
  • String instanceName – the given readable denotation of the operation desk
  • String version – the currently installed version of the client application
  • Date connectionDate – the last time this client application established a connection
  • Date lastActionDate – the last time this client application issued an action command (“was active”)

We can start all kinds of (justified) discussion about primitive obsession, too much information at one place and so on, but for this blog entry, only the categorization in master data and variable data is of interest. My opinion on the example is that the first three data fields (internalIdentifier, type and instanceName) are definitely in the master data category. The last two data fields are clearly variable data, while the version field is something in between. My guts tell me to categorize the version as master data, because it won’t change on a daily schedule.

When separating the two categories of data, the ClientInformation object may turn into a reference holder object only. In this case, the ClientInformation holds two references, one to a new ClientMasterData object (holding internalIdentifier, type, instanceName and version) and another one to a new ClientVariableData object (holding connectionDate and lastActionDate).

A less radical modification would be to let the master data remain in the ClientInformation object and only extract the variable data into a new ClientConnectionData object. If a client connects, only the referenced ClientConnectionData object has to change.

If you separate your master data from the variable data, you can very easily concentrate on the variable data for performance optimizations. This is where the data changes will happen and a tuned storage strategy will pay off. The master data should be designed more carefully concerning the type information, so if we really start the discussion about primitive obsession, I would first tend to the master data fields and argue that the type shouldn’t be a String but an Enum and the version should be a more sophisticated Version type. This could be modelled even with a slow object/relational mapper because the data is only written/read once.

The next time you come across one of your data model objects that contain more than two data fields, have a look at their categorization in master and variable data. Perhaps you can see a good reason to split the object.

Depth-first programmers

Depth-first programmers are always busy creating horribly complicated solutions that are somehow off the mark. Here’s why and what to do against it.

Just as there are at least two fundamentally different approaches for searching, namely depth-first and breadth-first search, there are also different types of programmers. The depth-first programmer is a dangerous type, as he is prone to yak shaving and reinvention of the wheel.

The depth-first programmer

Let me try to define the term of a “depth-first programmer” by a little (true) story. A novice java programmer should make some changes to an existing code. To secure his work, he should and wanted to write unit tests in JUnit. He started the work and soon enough, first results could be seen. But when he started to write his tests, the progress notifications stopped. The programmer worked frantically for hours and then days to write what appeared to be some simple data-driven tests.

Finally, the novice java programmer reported success and showed his results. He wrote his tests and “had to extend JUnit a bit to do it right”. Wait, what? Well, in JUnit, the test methods cannot have parameters, but the programmer’s tests needed to be parametrized. So he replaced the part of JUnit that calls the test methods by reflection with an “improved” algorithm that could also inject parameters. His implementation relied on obscure data structures that provided the actual parameter values and only really worked for his needs. The whole mess was nearly intangible, a big bloat and needed most of the development time for the unit tests.

And it was totally unnecessary once you learn about “Parameterized” JUnit4 tests or build light-weight data drivers instead of changing the signature of the test method itself. But this programmer dove deep into JUnit to adjust the framework itself to his needs. When asked about it, he stated that “he needed to pass the parameters somehow”. That’s right, but he choose the most expensive way to do so.

He exhibited the general behaviour of a depth-first programmer: whenever you face a problem, take the first possible solution to a problem you can come up with and work on it without evaluation against other possibilities. And continue on the path without looking back, no matter how long it takes.

Stuck in activism

The problem with this approach should be common sense. The obvious option isn’t always the best or even a good one. You have to evaluate the different possible solutions by their advantages and drawbacks. A less obvious solution might be far better in every aspect but obviousness. Another problem with this approach is the absence of internal warning signs.

Getting stuck is an internal warning sign every programmer understands. You’ve worked your way in a certain direction and suddenly, you cannot advance further. Something blocks your anticipated way of solving the problem and you cannot think of an acceptable way past it. A depth-first programmer never gets stuck this way. No matter how expensive, he will pursue the first thing that brings him closer to the target. A depth-first programmer always churns out code at full speed. Most of it isn’t needed on second thought and can be plain harmful when left in the project. The depth-first programmer will always report progress even when he needs days for a task of minutes. He is stuck in activism.

Progress without guidance

This isn’t a rant about incompetent programmers. Every good programmer knows the situation when you suddenly realize that you’re shaving a yak when all you wanted to do is to add a feature to the code base. This is your self-guidance system regaining consciousness after a period of auto-piloting in depth-first mode. Every programmer behaves depth-first sometimes.

This can be explained with the Dreyfus Model of Skill Acquisition. On the first stage, called “Beginner”, you are simply not capable of proper self-evaluation. You cannot distinguish between good and not so good approaches beforehands or even afterwards. Your expertise in the narrow field of the problem at hand isn’t broad enough to recognize an error even when you are working on the error yourself for prolonged times.

In the Dreyfus Model, a beginner needs external guidance. Somebody with more experience has to point out errors for you and formulate alternatives as clearly and specific as possible. Without external guidance, a beginner will become a depth-first programmer. We’ve all been there.

 Be a guide

The real failure in the story above was done by me. Instead of interacting with the novice java programmer after a few hours when I thought he should be done by now, I let him “advance”. I could have avoided the resulting mess by providing guidance and a few alternate solutions for the immediate problem. I would give an overview of the problem’s context and some hints about the general direction this task should be solved.

Every depth-first programmer works in a suboptimal environment. The programmer tries his best, it’s really the environment that could do better.

So, the next time you see somebody working frantically on a problem that should be rather easy to solve, lend him a hand. Be gentle and empathic about his attempt and work with proposals, not with instructions. Perhaps you’ve spared yourself a mess like an unnecessarily extended JUnit library and the depth-first programmer the frustration when his hard work of several days is silently discarded.

A review of the year 2011 at Softwareschneiderei

This is a review of the year 2011 for the Softwareschneiderei, a software development company from Karlsruhe, Germany.

The current year 2011 is coming to an end. This is the traditional time to pause and reflect on what has happened. This blogpost tries to sum up our year in software development at Softwareschneiderei. It was an interesting, entertaining and successful year for us, that’s for sure.

The official parts

Our developer blog was alive throughout the hardest times of the year, when everyone was under full project load. Every week, one of our developer shares a little posting with the world. The blog is still managed by token only. Looking at the visitor statistics, we fully appreciate your attention. The first blog post of this year looked at the remainder of a failed project. “A tale of scrap metal code” was a detailed vivisection in three parts. Over the course of the year, we wrote about bogus error messages, Groovy, Grails, GORM and some confessions about coding style and multithreading. If you have the time, spend a few minutes to browse our blog post archive for this year.

Our “official” company blog, written in german language, had no activity this year. We can certainly do better than this and take it on the list for 2012.

The company homepage, written in german language, had continuous updates and extensions this year. We coupled the “Open Source Love Day” (OSLD) with our “Homepage Comittee”, when every employee has to improve the homepage in some aspects and present the change to the “comittee”. Unfortunately, this somehow lead to fewer OSLDs this year.

The ongoing Dev Brunch sessions thinned out a bit in the second half of the year. This was a concession to the ever-growing workload. We strive to establish a tighter schedule with accompanying blog posts in the next year.

The internal parts

We were under heavy development load this year. This isn’t a bad thing, but impacts the internal communication and team building process. We tried to cope a bit by restructuring the Open Source Love Day to a “Team Day”, when the whole team meets and works on various internal or hobby projects. Some of these days were spend on Code Camps and other training events. This means less love for the open source community, but crucial together time for us.

We picked up several “new” programming languages this year. You can tell by the blog posts that we worked with Python, Ruby, Flex/ActionScript and even VisualBasic on real projects. The VisualBasic experience was a little epiphany that it’s really the developer and not the language that leads to shitty software.

One method of continuous improvement is our “creativity budget”. Basically, it’s money every developer can spend to improve his workplace. This budget wasn’t used at all this year, as the workplaces seem to be optimal. This cannot be true, so we bought brand new computers or a big RAM upgrade for everyone. And every computer has a big enough SSD now. We took our own advice seriously and invested in our productivity.

Our developer crew grew again this year. We are beginning to think about the remaining space in the new office again. But as usual, we grow slowly and deliberately. There’s nothing worse than a team of strangers.

Conclusion

The year 2011 was great! We’re looking forward to the year 2012, with our motto of christmas 2011: “cheery and spry” (the original motto is in german language “froh und munter”, I hope the translation caught the original spirit).

Have a great turn of the year, everyone. We’d love to see you again next year.

The Story of a Multithreading Sin

The story of a bug that was caused by a common multithreading pitfall, the dreaded liquid lock.

In my last blog entry, I wrote about multithreading pitfalls (in Java), and ironically, this was the week when we got a strange bug report from one of our customers. This blog entry tells the story of the bug and adds another multithreading pitfall to the five I’ve already listed in my blog entry “When it comes to multithreading, better be safe than sorry”.

The premise

We developed a software that runs on several geographically distant independent “stations” that collect a multitude of environmental measurement data. This data is preprocessed and stuffed into data packages, which are periodically transferred to a control center. The software of this control center, also developed by us, receives the data packages, stores them on disk and in a huge database and extracts the overall state of the measurement network from raw data. If you describe the main task of the network on this level, it sounds nearly trivial. But the real functionality requirements are manifold and the project grew large.

We kept the whole system as modular as necessary to maintain an overall grasp of what is going on where in the system and installed a sufficient automatic test coverage for the most important parts. The system is still under active development, but the main parts of the network are in production usage without real changes for years now.

The symptoms

This might explain that we were very surprised when our customer told us that the control center had lost some data packages. Very soon, it turned out that the control center would randomly enter a state of “denial”. In this state, it would still accept data packages from the stations and even acknowledge their arrival (so the stations wouldn’t retry the transmission), but only write parts of the package or nothing at all to the disk and database. When the control center entered this state, it would never recover from it. But when we restarted the software manually, everything would run perfectly fine for several days and then revert back into denial without apparent trigger.

We monitored the control center with every means on our disposal, but its memory consumption, CPU footprint and threading behaviour was without noticeable problem even when the instance was in its degraded state. There was no exception or uncommon entry logged in the logfiles. As the symptom happened randomly, without external cause and with no chance of reversal once it happened, we soon suspected some kind of threading issue.

The bug

The problem with a threading issue is that you can’t just reproduce the bug with an unit or system test. We performed several code reviews until we finally had a trace. When a data package arrives, a global data processing lock is acquired (so that no two data packages can be processed in parallel) and the content of the package is inspected. This might trigger several network status changes. These change events are propagated through the system with classic observer/listener structures, using synchronous calls (normal delegation). The overall status of the network is translated in a human readable status message and again forwarded to a group of status message listeners. This is a synchronous call again. One of the status message listeners was the software driver for a LED ticker display. This module was a recent addition to the control center’s hardware outfit and used to display the status message prominently to the operators. Inside this LED software driver, some bytes are written to a socket stream and then the driver awaits an answer of the hardware device. To avoid the situation that two messages are sent to the device at the same time, a lock is acquired just before the message is sent. This code attracted our attention. Lets have a look at it:

private Message lastMessage = new Message();

public void show(Message message) {
    synchronized (this.lastMessage) {
        writeCommandAndWaitForResponse(Command.SHOW_TEXT, message.asBytes());
        this.lastMessage = message;
    }
}

The main problem here is the object the lock is acquired upon: the reference of lastMessage is mutable! We call this a liquid lock, because the lock isn’t as solid as it should be. It’s one of the more hideous multithreading pitfalls as it looks like everything’s fine at first glance. But this lock doesn’t have a complete “locking” effect because each caller may acquire the lock of a different instance. And a lock with a flawed locking behaviour is guaranteed to fail (in production). The liquid lock is like the bigger brother of the local lock. It isn’t local, but its mutability cause the same problems.

The bug finally turned out to be caused by the liquid lock in the LED display driver that got notified of system message changes when a data package arrived. But only if multiple messages were sent at once to the device, discarding some of the necessary answers in this circumstance or if the connection to the LED hardware would fail in the midst of a transmission, the system would not return from the write attempt. If one thread wouldn’t return to the data package processor, the global data processing lock would not be freed (read the start of this chapter again, this is the most important lock in the system!). And while the data processing lock was still held, all other data packages would be received, but piling up to obtain the lock. But the lock would never be returned from the thread waiting on an answer from a hardware device that had no intention to send another answer. This was when the control center appeared to be healthy but didn’t process any data packages anymore.

The conclusion

If you want to avoid the category of liquid lock multithreading bugs, make sure that all your lock instance references are immutable. Being final is an important property of lock instance references. Avoid to retrieve your locks from notoriously muteable data structures like collections or arrays. The best thing you can do to avoid liquid locks is to “freeze” all your lock instances.

Another insight from this story is that software modules have to be separated threadwise, too. It was a major design flaw to let the data processing thread, while holding the main processing lock, descend down into the deep ends of the LED driver, eventually getting stuck there for infinity. Some simple mechanisms like asynchronous listener notification or producer/consumer queues for pending transmission requests would have helped to confine the effects of the liquid lock bug inside the LED module. Without proper thread separation, it took down the whole software instance.