A tale of scrap metal code – Part II

The second part of a series about the analysis of a software product. This part investigates the source code and reveals some ugly practices therein.

In the first part of this tale about an examined software project, I described the initial situation and high-level observations about the project. This part will dive into the actual source code and hopefully reveal some insights. The third and last part will summarize everything and give some hints on how to avoid creating scrap metal code.

About the project

If you want to know more about the project, read the first part of this tale. In short, the project looked like a normal Java software, but unfolded into a nightmare, lacking basic requirements like tests, dependency management or continuity.

About the developer

The developer has a job title as a “senior developer”. He developed the whole project alone and wrote every line of code. From the code, you can tell his initial uncertainty, his learning progress, some adventurous experiments and throughout every file, a general uneasiness with the whole situation. The developer actively abandoned the project after three years of steady development. From what I’ve seen, I wouldn’t call him a “senior” developer at all.

About the code

The code didn’t look very repellent at first sight. But everywhere you looked, there was something to add on the “TODO list”. Let me show you our most prominent findings:

Unassigned constructors

The whole code was littered with constructor calls that don’t store the returned new object. What’s the point in constructing another instance of you throw it away in the next moment, without ever using it? After examining these constructors, it became apparent that they only exist to perform side effects each. The new object is registered with the global data model while it’s still under construction. It was the most dreadful application of the Monostate design pattern I’ve ever seen.

Global data model

Did I just mention the global data model? At the end of the investigation, we found that the whole application state lives in numerous public static arrays, collections or maps. These data fields are accessible from everywhere in the application and altered without any protection against concurrent modifications. These global variables were placed anywhere, without necessarily being semantically associated with their enclosing class. A data model in the sense of some objects being tied together to form an instance net with higher-level structures could not be found. Instead, different lookup structures like index-based arrays and key-based maps are associated by shared keys or obscure indices. The whole arrangement of the different data pieces is implicit, you have to parse the code for every usage. Mind you, these fields are globally accessible.

Manual loop unrolling

Some methods had several hundred lines of the exact same method call over and over again. This is what your compiler does when it unrolls your foreach loops. In this code, the compiler didn’t need to optimize. To add some myth, the n-th call usually had a slight deviation from the pattern without any explanation. Whenever something could easily be repeated, the developer pasted it all over the place. Just by winding up the direct repititions again, the code migth shrink by one quarter in length.

Least possible granularity

Just by skimming over the code, you’d discover plenty of opportunities to extract methods, raising the level of abstraction in the code. The developer chose to stick with the least possible granularity, making each non-trivial code a pain to read. The GUI-related classes, using Swing, were so bloated by trivialities that even a simple dialog with two text fields and one button was represented by a massive amount of code. Sadly, the code was clearly written by hand because of all the mistakes and pattern deviations. If the code had to deal with complex data types like dates, the developer always converted them to primitive data types like int, double or long and performed the necessary logic using basic math operators.

“This code is single-threaded, right?”

Despite being a Java Swing application, the code lacked any strategy to deal with multithreading other than ignoring the fact that at least two threads would access the code. We didn’t follow this investigation path down to its probably bitter end, but we wouldn’t be surprised if the GUI would freeze up occasionally.

“Exceptions don’t happen here”

If you would run a poll on the most popular exception handling strategy for this code, it would be the classic “local catch’n’ignore”. The developer dismissed the fact that exceptions might happen and just carried on. If he was forced to catch an exception, the catch block followed immediately and was empty in most cases. Of course, the only caught exception type was the Exception class itself.

“This might be null

One recurring pattern of the developer was a constructor call, stored in a local variable and immediate null check. Look at this code sample:

try {
    SomeObject object = new SomeObject();
    if (object != null) {
        object.callMethod();
    }
    [...]
} catch (Exception e) {
}

There is no possibility (that I know of) of object being null directly after the constructor call. If an exception is thrown in the constructor, the next lines won’t be executed. This code pattern was so prevalent in the code that it couldn’t be an accidental leftover of previous code. The accompanying effect were random null checks for used variables and return values.

Destabilized dependencies

If there is one thing that’s capable of derailing every code reader, analysis tool and justified guess, it’s wildcard use of Java’s reflection capabilities. The code for this project incorporates several dozens calls to Class.forName(), basically opening up the application for any code you want to dynamically include. The class names result from obscure string manipulation magic or straight from configuration. It’s like the evil brother of dependency injection.

Himalaya indentation

Looking at the indentation depths of the code, this wasn’t the worst I’ve ever seen. But that doesn’t mean it was pleasant. Like in Uncle Bob’s infamous “A crap odyssey”, you could navigate some classes by whitespace landscape. “Scroll down to the fifth crest, the vast valley afterwards contains the detail you want to know”.

Magic numbers

The code was impregnated with obscure numbers (like 9, 17, 23) and even more bizzare textual constants (like “V_TI_LB_GUE_AB”) that just appeared out of nowhere several times. This got so bad that the original author included lengthy comment sections on top of the biggest methods to list the most prominent numbers alongside their meanings. Converting the numbers to named constants would probably dispel the unicorns, as we all know that unicorns solely live on magic numbers(*). Any other explanation escapes my mind.

(*) On a side note, I call overly complex methods with magic numbers “unicorn traps”, as the unicorns will be attracted by the numbers and then inevitably tangled up in the complexity as they try to make their way out of the mess.

Summary

This was the list of the most dreaded findings in the source code. Given enough time, you can fix all of them. But it will be a long and painful process for the developer and an expensive investment for the stakeholder.

To give you an overall impression of the code quality, here is a picture of the project’s CrapMap. The red rectangles represent code areas (methods) that need improvements (the bigger and brighter, the more work it will take). The green areas are the “okay” areas of the project. Do you see the dark red cluster just right the middle? These are nearly a hundred complex methods with subtle differences waiting to be refactored.

Prospectus

In part three, I’ll try to extract some hints from this project on how to avoid similar code bases. Stay tuned.

A tale of scrap metal code – Part I

The first part of a series about the analysis of a software product. This part investigates some aspects of general importance and works out how they are failed.

This is the beginning of a long tale about an examined software project. It is too long to tell in one blog post, so I cut it in three parts. The first part will describe the initial situation and high-level observations. The second part will dive deep into the actual source code and reveal some insights from there. The third part wraps everything together and gives some hints on how to avoid being examined with such a negative result.

First contact

We made contact with a software product, lets call it “the application”, that was open for adoption. The original author wanted to get rid of it, yet it was a profitable asset. Some circumstances in this tale are altered to conceal and protect the affected parties, but everything else is real, especially on the technical level.

You can imagine the application as being the coded equivalent to a decommissioned aircraft carrier (coincidentally, the british Royal Navy tries to sell their HMS Invincible right now). It’s still impressive and has its price, but it will take effort and time to turn it around. This tale tells you about our journey to estimate the value that is buried in the coded equivalent of old rusted steel, hence the name “scrap metal code” (and this entry’s picture).

Basic fact

Some basic facts about the application: The software product is used by many customers that need it on a daily basis. It is developed in plain Java for at least three years by a single developer. The whole project is partitioned in 6 subprojects with references to each other. There are about 650 classes with a total of 4.5k methods, consisting of 85k lines of code. There are only a dozen third-party dependencies to mostly internal libraries. Each project has an ant build script to create a deployable artifact without IDE interference. On this level, the project seems rather nice and innocent. You’ll soon discover that this isn’t the truth.

Deeper look

Read the last paragraph again and look out for anything that might alert you about the fives major failures that I’m about to describe. In fact, the whole paragraph contains nothing else but a warning. We will look at five aspects of the project in detail: continuity, modularization, size, dependencies and build process. And we won’t discover much to keep us happy. The last paragraph is the upmost happiness you can get from that project.

Feature continuity

You’ve already guessed it: Not a single test. No unit test, zero integration tests and no acceptance test other than manually clicking through the application guided by the user manual (which we only hoped would exist somewhere). No persisted developer documentation other than generated APIDoc, in which the only human-written entries were abbreviated domain specific technical terms. We could also only hope that there is a bug tracker in use or else the whole project history would be documented in a few scrambled commit messages from the SCM (one thing done right!).
The whole project was an equally distributed change risk. The next part will describe some of the inherent design flaws that prohibited changes from having only local effects. Every feature could possibly interact with every other piece of code and would probably do so if you keep trying long enough.
It’s no use ranting about something that isn’t there. Safety measures to ensure the continuity of development on the application just weren’t there. FAIL!

Project modularization

The six modules are mostly independent, but have references to types in other modules (mostly through normal java imports). This would not cause any trouble, if the structure of the references was hierarchical, with one module on top and other modules only referencing moduls “higher” in the hierarchy. Sadly, this isn’t the case, as there is a direct circular dependency between two modules. You can almost see the clear hierarchical approach that got busted on a single incident, ruining the overall architecture. You cannot use Eclipse’s “project dependencies” anymore, but have to manually import “external class folders” for all projects now. The developer has forsaken the clean and well supported approach for a supposable short-term achievement, when he needed class A of module X in the context of module Y and didn’t mind the extra effort to think about a refactoring of the type and package structure. What could have been some clicks in your IDE (or an automatic configuration) will now take some time to figure out where to import which external folder and what to rebuild first because of the cycle. FAIL!

Code size

The project isn’t giantic. Let’s do some math to triangulate our expectations a bit. One developer worked for three years to pour out nearly 90k lines of code (with build scripts and the other stuff included). That’s about 30k lines per year, which is an impressive output. He managed to stuff these lines in 650 classes, so the average class has a line count of 130 lines of code. Doesn’t fit on a screen, but nothing scary yet. If you distribute the code evenly over the methods, it’s 19 lines of code per method (and 7 methods per class). Well, there I get nervous: twenty lines of code in every method of the system is a whole lot of complexity. If a third of them are getter and setter methods, the line count rises to an average of 26 lines per method. I don’t want every constructor i have to use to contain thirty lines of code!

To be sure what code complexity we are talking about, we ran some analysis tools like JDepend or Crap4j. The data from Crap4j is very explicit, as it categorizes each method into “crappy” or “not crappy”, based on complexity and test coverage (not given here). We had over 14 percent crappy methods, in absolute numbers roughly 650 crappy methods. That is one crappy method per class. The default percentage gamut of Crap4j ends at 15 percent, the bar turns red (bad!) over 5 percent. So this code is right at the edge of insanity in terms of accumulated complexity. If you want to know more about this, look forward to the next parts of this series.

Using the CrapMap, we could visualize the numerical data to get an overview if the complexity is restricted to certain parts of the application. You can review the result as a picture here. Every cell represents a method, the green ones are okay while the red ones are not. The cell size represent the actual complexity of the method. As you can see, the “overly complex code syndrome” is typically for virtually all the code. Whenever a method isn’t a getter or setter (the really tiny dark green square cells), it’s mostly too complex. Additional numbers we get from the Crap4j metric are “Crap” and “Crap Load”, stating the amount of “work” necessary to tame a code base. Both values are very high given the class and method count.

All the numbers indicate that the code base is bloated, therefore constantly using the wrong abstraction level. Applying non-local changes to this code will require a lot of effort and discipline from an experienced developer. FAIL!

Third-party dependencies

The project doesn’t use any advanced mechanism of dependency resolution (like maven or ant ivy). All libraries are provided alongside the source code. This isn’t the worst option, given the lack of documentation.
A quick search for “*.jar” retrieves only a dozen files in all six modules. That’s surprisingly less for a project of this size. Further investigation shows some inconvenient facts:

  • Some of these libraries are published under commercial licenses. This cannot always be avoided, but it’s an issue if the project should be adopted.
  • Most libraries provide no version information. At least a manifest entry or an appended version number in the filename would help a lot.
  • Some libraries are included multiple times. They are present for every module on their own, just waiting to get out of sync. With one library, this has already happened. It’s now up to the actual classpath entry order on the user’s machine how this software will behave. The (admittedly non-present) unit tests would not safeguard against the real dependency, but the local version of the library, which could be newer or older.

As there is no documentation about the dependencies, we can only guess about their scope: Maybe the classes are required at compile time but optional at runtime? The best bet is to start with the full set and accept another todo entry on the technical debt list. FAIL!

Build scripts

But wait, for every module, there is a build script. A quick glance shows that there are in fact four build scripts for every module. All of them are very similar with minor differences like which configuration file gets included and what directory to use for a specific fileset. Nothing some build script configuration files couldn’t have handled. Now we have two dozen build scripts that all look suspiciously copy&pasted. Running one reveals the next problem: All these files contain absolute paths, as if the “works on my machine award” was still looking for a winner. When we adjusted the entries, the build went successful. The build script we had to change was a messy collection of copied code snippets (if you want to call ant’s XML dialect “code”). You could tell by the different formatting, naming and solution finding styles. But besides being horribly mangled, the build included code obfuscation and other advanced topics. Applied to the project, it guaranteed that no stacktrace from any user would ever contain useful information for anybody, including the project’s developers. FAIL!

Summary

Lets face the facts: The project behind the application fails on every aspect except delivering value to the current customers. While the latter is the most important ingredient of a successful project, it cannot be the only one and is only sustainable for a short period of time. The project suffers from the lonely superhero syndrome: one programmer knows everything (and can defend every design decision, even the ridiculous ones) and has no incentive to persist this knowledge. And the project will soon suffer from the truck factor: The superhero programmer will not be available anymore soon.

Prospectus

There are a lot of take-away lessons from this project, but I have to delay them until part three. In the next part, we’ll discover the inner mechanics and flaws of the code base.

Code Camp Experiences II

A review of our first company code camp using Code Retreats like Corey Haines would do. Short summary: It was a lot of fun and we learned a lot. Go try it out yourself!

Last friday, we held a Code Camp instead of an Open Source Love Day (OSLD). We reserved a whole day for the company to pratice together and share our abilities on the coding level. While this usually already happens every now and then with pair programming sessions, this time we all worked on the same assignment and could compare our experiences. And this comparability worked great for us. This article tries to summarize our setup and the outcome of the Code Camp

Setup of the Code Camp

We tried to imitate a typical Code Retreat day in the manner of Corey Haines. If you haven’t heard about Code Retreats, Corey or the software craftsmanship idea, you could read about it in the links. The presentation of Corey at the QCon conference about software craftsmanship is also a valuable watch.

There are some resources on the internet about how to run a Code Retreat event from the organizational and facilitator’s point of view. This material gave us a good understanding of the whole event, even though our setup was different, as we had no explicit facilitator and fixed workplaces, already prepared for pair programming usage. We didn’t invite external programmers to the event, so every participant was part of our development team. We had to end the event by 16 o’clock due to schedule conflicts and started at 9 o’clock, so our retreat count would be lower than 6 or even 7.

Basically, we tried to program Conway’s Game of Life within 45 minutes in pairs of two developers repeatedly. After the 45 minutes have passed (supervised by an alarm clock), we deleted the code and gathered for an iteration review of 15 minutes. Then, we started over again. This agenda should repeat throughout the day. No other activity or goal was planned, but we anticipated a longer retrospective meeting at the end of the day.

Execution of the Code Camp

The team gathered at 9 o’clock and performed setup tasks on the computers (like preparing a clean workspace). At 09:15, we held an introduction meeting for the Code Camp. I explained the basics and motives of Code Retreats and presented the rules for Conway’s Game of Life. The team heard most of the information for the first time, so nobody was particularly more experienced with the task or the conduct.

The first iteration started at 10 o’clock and had everybody baffled by the end of the iteration. The first retrospective meeting was interesting, as fundamental approaches to the problem were discussed with very little words needed for effective communication. Everybody wanted to move into the second iteration, which started at 11 o’clock.

At the end of the second iteration, two of the four teams nearly reached their anticipated goals. In the retrospective, the results were incredibly more advanced compared to the first iteration. This effect was similar to my first code camp: The second iteration is the breakthrough in the problem domain. Afterwards, the solutions are refined, but without the massive boost in efficiency compared to the other iterations except the first one.

We went to lunch early this day and returned with great ideas for the next round. After a short coffee break with video games, we started at around 13:45 for the third iteration.

The third iteration resulted in the first playable versions of the game. The solutions grew more beautiful and the teams began to experiment with their approaches, as the content-related task was mentally covered. This was the most productive iteration in terms of resulting software. But as usual, the code was deleted without a trace directly after the iteration. The iteration review meeting brought up a radically different approach on the problem as previously anticipated. This inspired every team for the fourth iteration.

In the fourth iteration, every team tried to implement the new approach. And every team failed to gain substantial ground, just like in the first iteration. The iteration review meeting was interesting, but we skipped another iteration in favor of the full retrospective of the Code Retreat.

Effects of the Code Camp

The Code Retreat iterations had great impact on our team. We discussed our impressions informally and then turned back to the formal retrospective questions as suggested by Alex Bolboaca:

  • How did you feel?
  • What have you learned?
  • What will you apply starting Monday?

The first question got answered by a “mood graph”, rising steadily from iteration one to three, with a yawning abyss at iteration four. This was another strong indicator that the iterations sort of restarted with iteration four.

The second question (“What have you learned?”) was answered more variably, but it stuck out that many keyboard shortcuts and little helpful IDE tricks were learnt throughout the day. We tracked the origin and propagation of two shortcuts and came to the result that one developer knew them beforehands, transferred the knowledge to the partner in the first iteration and both spread it further in the second iteration. By the end of the third iteration, everybody had learned the new shortcut. It was impressive to see this kind of knowledge transfer in such a clear manner.

The third question revolved around the coolest new shortcuts and tricks.

But we learned a lot more than just a few shortcuts. Most of all, we had a comparable coding experience with every other developer on our team. This isn’t about competition, it’s about personality. And we’ve found that the team works great in every combination. Some subtle fears of “being behind with knowledge” got diminished, too.

Future of the Code Camp

Everybody wants to do it again. So we’ll do it again. We decided to perform one Code Camp every three months. This isn’t too often to wear off, but hopefully often enough to keep our practice level high. We also decided to run dedicated Code Camps with external developers soon. The first event will happen in December 2010.

An advent of unconditional quality code

A four-week experiment dealing with conditional statements and how to avoid or replace them. Starting at the first advent, the experiment runs until christmas. We invite you to join and share your experiences.

This blog entry invites you to an experiment in code. It’s an experiment that runs four weeks and can be performed secretly even at your workplace. It might improve the way you think about conditional statements in an object oriented programming language. You don’t need any special hardware or setup, just the will to change your coding style a bit each week.

The experiment

Beginning with this year’s advent (a season of the christian religion), you are asked to omit one type of conditional statement each week while programming your regular code. The omitted statements add up, so that you have to spare four different statements in the week before christmas. There is no relation to christmas (or religion) other than it’s a four week period at the end of the year, which is the perfect timeframe for the experiment. And you might buy yourself a little present for christmas if you succeeded at the experiment (idea: a new programming book).

The four stages

For every stage, you are asked to write your normal code without a specific statement. It is perfectly valid to use semantically equivalent code constructs to achieve the same goal. This experiment is even more successful if you are creative and diversified in your variations of the original statement. Remember that the stages add up. On the fourth stage, you are asked to use none of the statements mentioned below.

  • Stage 1 (first week): Don’t use “else”
  • Stage 2 (second week): Don’t use the conditional operator “?:”
  • Stage 3 (third week): Don’t use “switch”
  • Stage 4 (fourth week): Don’t use “if”

You are not asked to change existing code to conform to these restrictions, except you need to work on the lines that contain the prohibited statements. You should apply the rules to your new code rigorously, though.

Explanation of stage 1 (Don’t use “else”)

This rule bans all the different occurrences of the else-branch to your if-statements. It includes every “else if” or “elsif” your programming language might provide. The rationale behind the rule can be found in the Object Calisthenics, rule #2 by Jeff Bay. Here is an explanation of it by Being Cellfish.

Explanation of stage 2 (Don’t use the conditional operator “?:”)

Elvis is dead. Let this resemblance to his hairdo rest for a week, too. It contains a hidden else statement that is restricted since stage 1. Another rationale is that the conditional operator isn’t very easy to read/grasp if stretched out a long line.

Explanation of stage 3 (Don’t use “switch”)

A switch (or case, or select) statement is nothing but a big if-else cascade. It’s handy sometimes, but can be replaced by a lookup table (like a hashmap) virtually everytime . In Martin Fowler’s book “Refactoring”, the switch statement counts as its own code smell category. You should try to live without it for a week. If you need inspiration, try this article on how to avoid it.

Explanation of stage 4 (Don’t use “if”)

Yes, you didn’t misread. There is a whole campaign that tries to avoid the if-statement altogether. Read their website for inspiration on how to survive this week. Maybe you might make new friends with polymorphism and some other implicit conditional structures. Remember, this is a short week just before christmas. Try it, you might be surprised how easy it looks with hindsight.

Ready, steady, go!

This experiment starts with the first advent at Sunday, 28.11.2010. Every stage lasts for one week and adds up to the previous stages. The experiment ends at christmas.

Good luck! And if you’re done with it, drop us a comment with your experiences.

Code Camp Experiences

Experiences gained when performing a two day code camp with a team.

Some weeks ago, I conducted a code camp with a team of twelve developers that build a software product together for years now. The team had already introduced sporadic code reviews (in the team vs. author review style), so the main emphasize of the meeting was to improve team coherence by writing code together while generally having some time off project. In this article, I describe what I had planned, what happened and what the effects are so far.

A plan for the code camp

The code camp was scheduled for two consecutive days when the whole team gathers in one room with one computer for each pair. We would switch pairs (and seats) after each iteration, with one iteration being 45 minutes coding time followed by 2-3 minutes presentation of the achievement to the team. With a recreation break of 20-30 minutes, this means one iteration every two hours.

Every iteration starts from scratch, without access to previous code fragments (see also the code retreat concept). This had several reasons: I wanted the iterations to be comparable. Some of the insights I wanted to share are dependent on directly assimilable experiences. The iterations should also be independent, without ballast from previous sessions.

On the first day, we started with a given code resembling a little puzzle game in Java Swing. The code worked, but had some bugs and was written in an awful manner. It was unknown code for the team with no emotional attachments. The assignment for each iteration was to refactor the code to something equivalently working, but much “better”. How this “better” is defined was up to the teams.

On the second day, we started with a blank editor and had the task to code the same little puzzle game (in Java Swing) we refactored the day before. Even with some practice, it was nearly impossible to finish within time, so concentration on the most important key aspects of the code was crucial. The main lesson here was to “create from scratch” rather than “fix the existing”.

What really happened

Day one

The camp started with the usual delay for setting up all the computers in a uniform manner. This couldn’t be prepared beforehands, as the computers were in use by another group. When we installed our software, we found the hotkey configuration of the whole system severely flawed (for example, Ctrl+1 was defined as “set keyboard layout to traditional chinese”).

To warm up for the first iteration, I presented the existing code and explained its structure in detail. The code was slightly too much to remember it all in one pass, so only a rough understanding remained. Every team had to examine the code again during their work.

After the warm up, the first iteration started, with everybody buzzing over the code. The 45 minutes went by really fast and the first presentations focussed on local improvements. No team had restructured the code in any meaningful way, but every solution was perceived as “better” than the original. One team failed to get their refactored code to work again.

The second iteration held the biggest surprise of the whole camp. The 45 minutes flew by and the presentations showed the difference. One team failed to work on the assignment, but every other team presented a solution that was far superior of the original code. Some teams restructured the code to an extend where the original structure wasn’t recognizable anymore. The distinction between the first and the second iteration was so great that everybody was baffled by what could be achieved in 45 minutes when you do it for the second time.

The third iteration added some interesting twists on the best solutions of the second iteration, but didn’t produce the massive boost in productivity and code quality. Everybody felt worn out afterwards, so we decided to close the coding part of this day.

While working with the Java Swing code, nobody on the team noticed the threading flaws in the code. When I pointed this out, I was asked to explain the mechanics of the Swing threading model. The team develops a web application and hasn’t much exposure to desktop application development, let alone with Java Swing. So we ended the day with a lecture about the EDT, the EventQueue and the SwingWorker.

The whole team strolled to a bar to share some beers afterwards.

Day two

After a short night, we gathered early in the morning to continue the coding part of the camp. I explained the task (develop the game from scratch) and off we went.

The first iteration yielded only very rudimentary results. One team started with an UML diagram of the application structure and had to stop after setting up the outline of every method in code. Most other teams started with the domain model and failed to attach the GUI part of the application. All solutions had similar concepts in mind, no team used test driven development or other “advanced” techniques.

As a result of the poor performance, we decided to change the rules. Instead of scrapping the whole code, every new team could take over the code base of any other team, as long as it wasn’t the own. In the second iteration, we completed the drafted solutions of the previous team. This didn’t work out, too. The teams were frustrated by their lack of results.

We changed the plan again and held a prolonged review discussion of the code camp instead of a third iteration. This was by far the better choice with hindsight.

The effects

The code camp was perceived very positively by the attendees. The main goal of the camp was not about coding, but about team coherence and team focus. We learnt a lot about the personal style and abilities of each team member because the code samples shown in the code review were directly comparable. And we revealed team problems we weren’t aware of yet, but some problems we thought would arise did not. This was a very healthy process, because some of these issues can be addressed directly now.

The side benefit of the camp, as stated by one programmer was the increased awareness that “throwing away your code and starting over isn’t as hurtful as I thought”.

Every attendee stated that they want another code camp soon.

Personal summary

The code camp greatly improved my sense for the team and for the individual team members. By sharing a common code base and performing the same tasks, I could directly see (and feel) their thoughts and abilities. The camp is a powerful way to get to really know a team. If you have to mentor a whole new team, consider performing a code camp to get in touch with them.

Why I give lectures in software engineering

I’m often asked why I give lectures in software engineering, as they appear to not pay off for me. I think they do and here is why.

<a href="http://de.fotolia.com/id/21746212" mce_href="http://de.fotolia.com/id/21746212" title="" alt="">falcn</a> - Fotolia.com

For more than eight years, I give lectures in software engineering, object oriented programming and software development “best practices”. I have to spend nearly a day every week for six months in the year to prepare and hold the classes. My normal work schedule is always very stuffed with tasks, I have to affront my other duties sometimes in order to show up in front of the students. On many occassions, I’ve been asked why I keep giving lectures despite pressing liabilities, inferior payment and generally better alternatives elsewhere.

Here is a list with answers I’ve given to this questions over and over again. I do not want to convince you that giving lectures is the best thing right after sliced bread or that you will experience any of these if you manage to get in the same position. It’s just a rational explanation why the question still strikes me as odd.

  • It’s pure fun – This surely doesn’t count for everyone, but for me, speaking (ranting, raving, arguing) about software development counts as fun times. Being “on stage” in front of the students helps me to free my thoughts from dead freight and completely concentrate on the topics.
  • I’m being paid to recapitulate the basics – This are two advantages in one: being paid cannot be bad (albeit payment can always improve) and to repeat the basics of my craft on a regular schedule can be seen in the tradition of katas. I’m very bulletproof in discussions about fundamental topics of software engineering because I’ve heard most questions and had to answer them multiple times already.
  • I’m constantly learning new facets about well-known topics – My students always bring in unique and original thoughts about topics that I thought to have mastered. And then, a new way to access things emerges, at least for me. I feel very certain that I’m still learning more during the lectures than my students do. And feedback suggests that they learn a lot.
  • I’m honing my verbal abilities – Giving a lecture is all about speaking without script and responding to the audience. You have to make your points, but you cannot force them. Sometimes I feel like a stand-up comedian for technical knowledge. Having the ability to speak fluently while preparing the next topic in the back of your head is a great advantage in every situation including verbal communication.
  • Roughly 100 aspiring developers remember me every year – What they will remember me for can be debated about, but they will remember. This is all about “networking”, but focussed on members of my own profession. The reach of this network amazes me every time when it loops back.
  • I can contact every local company with job training – Due to the nature of the Cooperative State University where I’m giving my lectures, I can also establish contact to every software company in the vicinity. Many contacts would never happen without my function as lecturer.
  • I keep in touch with hypes – Students are easy prey for IT hypes. Their experience with different technologies isn’t embittered by analogies from the past. All I have to do is to listen to them when they tell me about their work and hobby projects. And then I can draw my own conclusions based on their first-hand experience.

All these reasons and some more are enough for me to stick with the job. You can see a lot of short term benefits and some aspects that might pay off at medium term. On the long run, I’m convinced that my personal advantages from this job will outweight the (sometimes serious) drawbacks. And then, I haven’t yet included the advantages that my students took along from my lectures, hopefully.

If you happen to give lectures too, I would be pleased if you blog about your reasons for doing so, and announce your post here. Or just use the comment section.

Java Swing Layouting done right

A praise of the most developer-friendly Java Swing layout manager to date: DesignGridLayout.

Layout Managers were an huge benefit for Java Swing. They enabled software developers to program layout rather than to “drag and drop” it with some proprietary GUI builder. That’s nothing against a good GUI builder, but against the “source code” that gets generated as a result of using it. But after some time of playing and working with the layout managers given by Swing itself, we concluded that they weren’t up to the task. Since then, we were constantly on the lookout for new and better ways to tackle the layouting task.

A history of layout managers

Let’s reiterate our major path with different layout managers:

  • GridBagLayout – the most versatile layout manager included in the Java Swing core classes. It’s capable to handle virtually every layouting task, but the price is huge constraint setup code. Since the code bloats with even facile complexity in the dialog, it’s not maintainable once written. The advantages over GUI builders aren’t really present.
  • StringGridBagLayout – has the same power as GridBagLayout, but with much more concise constraint definitions. It uses a string based domain specific language that you have to learn. After a while, you begin to feel a clumsiness when inserting variables into the constraints.
  • TableLayout – was a new approach to layouting by applying a global grid to your panel. You define the grid by specifying row and column constraints. If you need special cell constraints afterwards, you can alter them, but it’s getting bloated again.
  • StringTableLayout – provided a string based domain specific language over the TableLayout. It had some nice additional features, but lacked versatility with dynamic GUIs.
  • FormLayout – was a great relief and a good companion for many full sized layouting tasks. By concentrating on a problem domain (form based layouts), it played out some advantages over general purpose layout managers. This layout is still in use here.
  • MigLayout – the bigger brother of all these layouts. MigLayout comes with several pages of cheat sheets and you’re soon lost without it. It combines the approaches of all layout managers listed (and many more) and blends them into a massively powerful and versatile product. If you learn this layout manager thoroughly, you’ll never have to look elsewhere. But the learning curve is steep and the complexity of your code scales with the complexity of the GUI (which isn’t a drawback).

All these layout managers added value to our GUIs and are in use until today, albeit seldom.

Keep it simple

Most of the time, your dialogs aren’t these super-fancy, highly dynamic full-page layouts every UI designer dreams about. If they are, pick one of the layout managers from the list and wade through the constraint setup. But let’s say you want to layout a rather plain dialog with some widgets, but you want to do it quick without sacrificing the looks. Here is a developer-friendly solution for this task: Use the DesignGridLayout manager.

Slick and easy layouts

The one thing that differentiates the DesignGridLayout from almost every other layout manager is that you use the layout manager instance itself (in a fluent interface style) to arrange the constraints of your grid. You do not add your widgets to the panel and hope for the layout manager to catch up with the layout, you add them to the layout manager (and hope for it to fill it into your panel, which it does nicely). Here is a little example of the API usage:

JPanel content = new JPanel();
DesignGridLayout layout = new DesignGridLayout(content);
JTextArea history = new JTextArea();
history.setRows(5);
JTextField message = new JTextField();
JButton sendNow = new JButton("Send");
layout.row().grid(new JLabel("History:")).add(new JScrollPane(history));
layout.row().grid(new JLabel("Message:")).add(message, 2).add(sendNow);
content.setLayout(layout);

If you are interested in the possibilities of the layout manager, you should read the usage introduction page of DesignGridLayout.

Developer-friendly approach

One big advantage of the fluent API when compared with the string based constraint definitions is the compiler and type system support. You can’t spell anything wrong and the code completion feature of your IDE guides you to the right method and parameter order. The other advantage is that you don’t need to mess with pixel sizes for spacing and such. It’s handled by the layout manager in the most comfortable manner.

And because an article about a layout manager isn’t of any worth without a picture, here’s one:

This is a frame with the panel we constructed in the example code above.

Are programming books overrated?

A little insight gathered through feedback from an internship. Software development books are somewhat overrated as they can’t teach practice well.

In the last few weeks, we had an internship of a student that just finished academic high school (“Gymnasium”) and is looking forward to take up studies in computer science. He wanted to get in touch with the practical aspects of the career he is about to choose. The programming courses in school merely covered the basics of a programming language (Java) and some UML.

We prepared the student for the internship by feeding him several books we thought were appropriate for his level of knowledge. The books were a beginner’s book about Java (Head First Java), an introduction to unit testing (Pragmatic Unit Testing) and a foundation on clean code programming (Refactoring). Our student read them thoroughly and could make references to the chapters during pair programming sessions.

Retrospective on the books

But one feedback we got from him was that the books alone were nearly useless for his case. If there wouldn’t have been tutorial style pair programming coding sessions and several short lectures , he couldn’t grasp the deeper meaning of the book chapters he read (he suffered from the “blank slate blockade” several times). This came a bit as a surprise for us, as the student was very clever and really into it. It wasn’t the student, it was the books.

But you can’t blame it on “Refactoring”, for example, as this book is an all-time classic filled with really important knowledge. It has to be the medium itself, books are not the ideal source to learn about programming and software development.

Books are part of the academics

There is an old question in our profession. It revolves around if we are more like engineers or artists, craftsmen or scientists. In the core of this question is a uncertainty about the right model of education. Artists and craftsmen prefer more practical training, with apprentice/master relations and personal knowledge transfer. For engineers and scientists, literature and more standardized lectures are better suited. Academic knowledge is transferred during debate, not during exercises.

The duality of our profession

Projecting the feedback of our student onto this question, there seems to be a duality in our profession: Both (or all four if you want) approaches are needed to form a whole. You can’t learn the theory and expect to excel on the job. But pratical experience alone will not suffice to keep up with the pace of our profession. Good books are like afterburners here, you’ll be hurled forward by every page.

Conclusion

If it’s really true that we need to learn our profession both ways at once, pair programming (in the tour guide or backseat driver style) is an essential part of our qualification. And our current university curriculum fails to deliver this part. Students nowadays can team up to program together on an assignment, but that’s not learning from a master (unless one in the team has distinctly more experience than everybody else and is able to transfer it). So I vote to bring more craftsmanship to the academic education, as the books alone won’t cut it.

Your opinion?

What’s your opinion on this topic? Drop us a line about your thoughts.

Add flair to your code: Code Squiggles

Introduction to Code Squiggles, a coding style that improves the readability of your java code.

Wrought Iron by quadriremeFor several months now, I’m experimenting with a programming style that you might want to call “syntax aware programming”. Every coding step starts with the question “what do I want to achieve and how do I want to write it down?”. Then I proceed to write it down in this perfect manner, mostly some english sentence, and try to incrementally convert the syntax into something the java compiler stops complaining about. There’s a lot to be learnt about API design, naming and creative syntax usage this way. One thing I’ve discovered along the way are Code Squiggles.

Introduction to Code Squiggles

Code squiggles are little methods that contain no functionality, other than directly returning the single given parameter. Their purpose is to make the code more fluently readable by casual readers. Calling these methods is absolutely optional, as they offer no behaviour at runtime. But by bloating your code with these method calls, you lower the amount of syntax rules and conventions the reader has to know before being able to understand the code. The process of adding the method calls feels like adding “happy little squiggles” (I certainly miss Bob Ross!) to your code.

Adding Code Squiggles by example

Let me give you a full example how Code Squiggles can turn your boring old java code into something even non-programming readers can grasp without problems. Note that the process of transforming the code isn’t the process I initially described, but the way to deal with existing code.

The initial code fragment looks like this:

assertTrue(filesDirectory.getChild("0.png").isFile());

As you can see, it’s an assertion line of an unit test. We improve the readability by extracting the intent of the assertion into the called method name (it’s a normal “extract method” refactoring):

assertFileExists("0.png", filesDirectory);

Now is the time to add the first Code Squiggle:

assertFileExists("0.png", in(filesDirectory));

You’ve noticed the difference? It’s just the word “in”, but it improves the semantics of the second parameter. As I promised, Code Squiggles are methods without any functionality worth talking about:

protected VirtualFile in(VirtualFile filesDirectory) {
    return filesDirectory;
}

The method takes a parameter and returns it. The real “functionality” of this method lies within its name. Everything else is only necessary to overcome (or overcode) java’s shortcoming of advanced syntax definition measures.

That’s all about Code Squiggles. But it doesn’t stop here. Let’s improve the example to its final shape, when we add two squiggles:

assertFile(withName("0.png"), existsIn(filesDirectory));

Read this line out loud! And notice the subtle change in the assertion method’s name. It begins to deminish clarity without the Code Squiggles. That’s when your API begins to depend on them. But it’s still perfectly valid java code if you just omit them.

The conceptual origin of Code Squiggles clearly comes from the behaviour driven development (BDD) style of writing tests and the ScalaTest Matchers. So I can’t claim much originality or cleverness myself here. But Code Squiggles, despite this initial example, aren’t limited to test code.

Adding inline documentation with Code Squiggles

Remember the last time you needed to integrate an horrible third-party API? Probably, there was a method with seven or eight parameters and all of them were primitive ints. No matter how often you called this method, you couldn’t remember the order of these ints. That’s when Code Squiggles might help you a bit.

This is the signature of a fairly complex method:

protected int performCalculation(int value, int lowerLimit, int upperLimit, int offset) { ...

Therefore, your call isn’t very self-explanatory:

int result = performCalculation(10, 2, 23, 13);

But it might be a lot more understandable when you add some squiggles:

int result = performCalculation(value(10), lowerLimit(2), upperLimit(23), offset(13));

I won’t spell out the squiggle implementations, as they should be straight-forward. Note that the java compiler doesn’t catch up here. You can easily swap the squiggles around to obfuscate your code. All you got is read-time safety. If you want compile-time safety, you need to replace the primitives with types and probably pay for some rounds of beer for the third-party API developers to include your changes or write an adapter (“corruption layer” is my preferred term here).

Regular use of Code Squiggles

If you want to use squiggles a lot, you might think about collecting them in a dedicated class. Design them to be static and generic, and you can use them easily with static imports:

public static <T> T existsIn(T instance) {
    return instance;
}

But remember that there are reserved keywords in java that might hamper you a bit.

Conclusion

Code Squiggles are useless bloat to your code unless you value read-time safety or casual readability. They can tidy up your code to a point where method or even class names are affected to form a block of code where you only have to replace the funnier chars with blanks to obtain a paragraph of plain written english anybody can read and understand.

What are your ideas towards Code Squiggles? Have you used them on your code? Mind to share the result?

Code squiggles are little methods that contain no functionality, other than directly returning the single given parameter. Their purpose is to make the code more fluently readable by casual readers.

Review of the Sensor+Test 2010 Measurement Fair

A review of our visit to the Sensor+Test 2010 Measurement Fair in Nuremburg, Germany. The most suprising discovery was a Java Virtual Machine running on an eight bit microprocessor, offering threading, garbage collection and exception handling.

Two weeks ago, we visited the german Sensor+Test 2010 Measurement Fair in Nuremburg. It’s a central trade fair for the sensor technology sector in Germany and Europe. We as software developers weren’t directly interested in the sensor hardware that was presented to the qualified visitors. We didn’t take part in the technical conference that was held in parallel. Why we were there and what we’ve found is the topic of this blog post.

General impressions of the trade fair

The sensor industry has a strong orientation towards classical technology and engineering. It’s not “new economy”, as you can tell from the exhibition booths and their setup. Most booths look like they have been used for several years with variations only in details. Most booth personal are engineers or managers of the companies presenting their latest products. It’s unlikely to talk to a sales person, several booths even had a paper sign saying “salesman needed” attached nearby. The industry knows very well what it is selling and capable to promise and achieve. We didn’t spot a single “booth bunny” on the whole trade fair, which is in great contrast to some high-strung events of other industries. Instead, you could walk into a booth and talk to the engineers without having to pass several layers of sales representatives.

Impressions of the exhibited technology

Miniaturization is a key trend for the hardware part of the sensors. Some display cabinets would have gone through as empty if there wasn’t a descriptive plate pointing to the dust particles that resemble whole sensor arrays. Energy efficiency and the new trend of energy harvesting, utilizing even minor differences in temperature or other physical quantities to power a device are this year’s buzzwords, too.

Why we were there

Our goal was to find out about the software side of the sensors. We wanted to grasp the software technologies used to drive the hardware and to process the data. We wanted to know what to expect when “technical software products” are exhibited. And we wanted to find out about the depth of knowledge about software development in these companies.

What we found out

  • Technical software products aren’t pretty. We’ve known this before, but had some hopes for stylish new user interfaces with Apple-like multitouch screen gesture controls and graphical effects. Instead, we’ve seen the same old 16-color line charts oscillating on a screen that contains all the buttons and handles that the program can offer. The graphical level was comparable to plain Windows 95 dialogs. Even when new-era Office ribbons were used, the main content stayed the same, posing an even bigger contrast. One thing that bugs me about technical software is the common existence of a big red “exit/quit/leave/stop” button on the main screen. I wonder where that might come from (see picture for hints).

  • The software platform in use is legacy Microsoft products. That’s not a problem for us, we are polyglot programmers. But only a few booths had other operating systems booted or an Apple in use. We asked one of them for visitor feedback on the Apples, they got told that their choice of hardware was “exotic, but likeable”. We didn’t find any Windows 7 in use on the whole trade fair.
  • Most sensor companies know a lot about software development tools. Anticipating the level of detail the engineers know about specific programming languages or tools from their rather monocultural choice for their products fails. Topics like Java, Eclipse, Smartphone programming, Android, etc. aren’t news to them. For example, in the recent professional journals of their sector, they discuss the different languages and platforms for smartphone software development.
  • Most sensor companies are desperate for embedded device programmers. We didn’t find a booth that wasn’t interested in finding conventional embedded device programmers (mostly a domain for C language experts with a good grasp of hardware design). So if you happen to be one, scan the exhibitor list and make some phone calls.

What we discovered

Somewhere in a corner of the second hall, in one of the last booths before the press center, there was a stand with big posters stating “Java on an 8-bit microcontroller” and “Exceptions, Threads, Garbage Collection within 60 kByte”. The whole buzzwords seemed a bit out of place, given that the booths around were announcing “better energy efficiency” or “greater effective range”. But it was a pleasant surprise for us. The german company calls itself Virtenio and is a new-founded spin-off of the Technische Universität Berlin. They achieved to develop a Java Virtual Machine that fits the teeniest ARM processor while still offering some performance and a good level of comfort for the programmers. It’s a very promising combination of embedded device and platform for “normal” software developers.

Summary of our visit

Our visit of the Sensor+Test 2010 Measurement Fair did pay off in many directions. We good our questions answered, discovered some new technologies and got very much inspiration about what’s going on in the embedded device sector. We are looking forward to be there again next year.