Make friends with your compiler

Suppose you are a C++ programmer on a project and you have the best intentions to write really good code. The one fellow that you better make friends with is your compiler. He will support you in your efforts whenever he can. Unless you don’t let him. One sure way to reject his help is to switch off all compiler warnings. I know it should be well-known by now that compiling at high warning levels is something to do always and anytime but it seems that many people just don’t do it.
Taking g++ as example, high warning levels do not mean just having “-Wall” switched on. Even if its name suggests otherwise, “-Wall” is just the minimum there. If you just spend like 5 minutes or so to look at the man page of g++ you find many many more helpful and reasonable -W… switches. For example (g++-4.3.2):


-Wctor-dtor-privacy: Warn when a class seems unusable because all the constructors or destructors in that class are private, and it has neither friends nor public static member functions.

Cool stuff! Let’s what else is there:


-Woverloaded-virtual: Warn when a function declaration hides virtual functions from a base class. Example:

class Base
{
public:
virtual void myFunction();
};

class Subclass : public Base
{
public:
void myFunction() const;
};

I would certainly like to be warned about that, but may be that’s just me.


-Weffc++: Warn about violations of the following style guidelines from Scott Meyers’ Effective C++ book

This is certainly one of the most “effective” weapons in your fight for bug-free software. It causes the compiler to spit out warnings about code issues that can lead to subtle and hard-to-find bugs but also about things that are considered good programming practice.

So suppose you read the g++ man page, you enabled all warning switches additional to “-Wall” that seem reasonable to you and you plan to compile your project cleanly without warnings. Unfortunately, chances are quite high that your effort will be instantly thwarted by third-party libraries that your code includes. Because even if your code is clean and shiny, header files of third-pary library may be not. Especially with “-Weffc++” this could result in a massive amount of warning messages in code that you have no control of. Even with the otherwise powerful, easy-to-use and supposedly mature Qt library you run into that problem. Compiling code that includes Qt headers like <QComboBox>with “-Weffc++” switched on is just unbearable.

Leaving aside the fact that my confidence in Qt has declined considerably since I noticed this, the question remains what to do ignore the shortcomings of other peoples code. With GCC you can for example add pragmas around offending includes as desribed here. Or you can create wrapper headers for third-party includes that contain


#pragma GCC system_header

AFAIK Microsoft’s compilers have similar pragmas:


#pragma warning(push, 1)
#include
#include
#pragma warning(pop)

Warning switches are a powerful tool to increase your code quality. You just have to use them!

Drawing your background with Javascript

Having complex backgrounds (like radial gradients) or even dynamic backgrounds in a web page is not an easy task using only images. Javascript helps here and the SVG/VML support in modern browsers makes drawing a reality. Looking around in the Javascript library world there are some which have APIs for crossbrowser drawing or charting. But if you are looking for a small and easy to learn on you have to search. We found raphaeljs, a project which started as a 20%er at Atlassian. It has an API which is close to SVG but also supports non SVG browsers like IE. How to make animations and complex drawings is easy to learn. The examples give a hint of what is possible.
We needed a radial gradient to have a smooth background, looking at the docs and the SVG spec, it was a four liner:

var Paper = Raphael("canvas", width, height);
var b = Paper.circle(width / 2, height / 2, Math.max(width, height));
var gradient = {type: "radial", dots: [{color: "#FFFFFF"}, {color: "#D5D5D5"}], vector: [0, 0, 0, "100%"]};
b.attr({"gradient": gradient});

About String Concatenation in Java or “don’t fear the +”

When it comes to string concatenation in Java many people have almost religious views about performance and style. Sadly, there are some misconceptions and misinformation especially about the performance bits. Many people think that concatenating many strings using + means expensive string copying each time and is thus slow as hell which is mostly wrong.

Justin Lee has a nice writeup of the most prominent concatenation options. But imho he misses out some things and his benchmark is a bit oversimplified although it does tell a true story. I assume that he followed at least the basic rules for performance measurement as his results suggest.

Now I want to try to clarify some points I think he missed and I find important:

  • Concatenation using + in one statement is actually compiled to the use of StringBuilder (at least for Sun Java6 compilers, where I checked it in the debugger, try it yourself!). So it’s no surprise that there is no difference between these two options in Justin’s benchmark.
  • It should be clear that the format variants have some overhead because they actually do more than just concatenate strings. There is at least some string parsing and copying involved so that these methods should be used for the cases where for example parameter reordering (think I18N) is needed or readability suffers using normal concatenation.
  • You have to pay attention when using + concatenation over the course of multiple statements because it then involves string copying. Consider the following code: Critical String Concatenation Here it really does make a difference which option you choose. The StringBuilder will perform far better for higher loop counts. We had a real world issue back some time with that when we used the Simple web framework for serving directory listing of several thousand files. The HTML-code was generated using a concatenatePlus()-style method and took like 40(!) seconds. After changing the code to the StringBuilder variant the page was served in sub-second time.

Whether you use + or StringBuilder is mostly a matter of taste and readability in many cases. When your string concatenation gets more complex you should really consider using StringBuilder as it is the safe bet.

On teaching software engineering

Be sure to include current topics in your lectures when teaching software engineering. Here are some hints.

overheadIn my rare spare time, I hold lectures on software engineering at the University of Cooperative Education in Karlsruhe. The topics range from evergreens like UML to modern subjects like aspect oriented programming (AOP) or Test Driven Development (TDD).

One thing I observe is that students don’t have difficulty separating the old topics from the current ones, even if they hear both of them for the first time. It seems that subject matter ages by itself, just like source code does. So, I’m constantly searching for new topics to include in the lectures, replacing the oldest ones.

Three things to include in your lectures

Some months ago, I read a very good blog post written by Alan Skorkin, titled “3 Things They Should Have Taught In My Computer Science Degree”. Alan covers three points:

  1. Open Source Development
  2. An Agile Process (e.g. XP, Scrum)
  3. Corporate Politics/Building Relationships

The idea of missed opportunities to tell some fundamentals to my students struck me. I compared my presentations to the list, finding the leading two topics covered to a great extent. The last one, corporate politics, is a bit off-topic for a technical lecture. But nevertheless, it’s too important to omit completely, so I already had included some Tom DeMarco lessons in my presentations. Perhaps I can build this part up a bit in the future.

What they should have taught me

Soon afterwards, I though about things my lecturers missed during my study. Here’s the list with only two points in addition to Alan’s list:

  • Age and “maturity” of topic: When I was a student, I quickly identified old topics, like my students do nowadays. What I couldn’t tell was if a topic was mature (a classic) or just deprecated. It would have helped to announce that a topic was necessary, but of little actual relevance in modern software development craftmanship. Or that a topic is academical news, but yet unheard of in the industry and lacking wide-spread acceptance. Both extremes were blended together in the presentations, creating an unique mixture of antiquated and futuristic approaches. This is a common problem of Advanced Beginners in the Dreyfus model.
  • Ergonomics and Effectiveness: I still can’t believe I didn’t hear a word about proper workplace setup from my teachers. I had courses teaching me how to learn, but not a single presentation that taught me how to work. This topic ranges from the right chair over lighting to screen size and quantity and could be skimmed over in less than an hour. But it doesn’t stop with the hardware. Entire books like Neal Ford’s “The Productive Programmer” cover the software side of effective workplace setup. And even further, the minimal set of tools a software developer should use (e.g. IDE, SCM, CI, issue tracker, Wiki) wasn’t even mentioned.

I hope to provide all these topics and information to my students in a recognizable (and rememberable) manner. They deserve to learn about the latest achievements in software engineering. Otherwise you aren’t prepared to work in an industry changing fundamentally every five to ten years. Of course, hearing about the classic stuff is necessary, too.

Give me feedback. What are your missed topics during apprenticeship, study or even work?

—

Update: In case you can’t visit my lectures but want to know a bit about ergonomics, I’ve written two blog articles on this topic:

Lightweight dependency management

Managing project dependencies without maven or ant ivy, using a custom ant task to ensure classpath orthogonality.

Java’s classpath is a powerful concept – when used appropriate. As your project grows larger in terms of code and people, it gets harder to ensure that your classpath is correct. A great danger arises from JAR files containing different versions of the same resource. You might end up running different code than you think, leading to strange effects. If you build your classpath using wildcards, you can’t even control the order your JAR files are loaded.

Managing dependencies

To avoid the issues mentioned above, you need to manage your project dependencies. It’s a common practice to implement the build process of the project using maven or ant ivy. Both tools provide dependency mangement by declaration. But at a high cost. Especially maven has received some malice lately, criticizing its steep learning curve and complexity.

Scratching the biggest itch

We decided to try a different approach to dependency management, tackling only our biggest concern: The duplication of classpath resources. We take care of the scope of a third-party library, put required JARs in the repository (to us, third party binary artifacts are part of the project source) and update manually. The one thing we cannot assure manually is that every resource is unique. Sometimes, the same class is included in different JARs, as it seems to be common practice among java web frameworks.

Ant to the rescue

Thus, I wrote a custom ant task that, given the classpath, checks for duplicate entries. If it finds one, it lists the culprits and optionally aborts the build process. Included in our continuous integration system, it gets run every time somebody performs a change. You can’t forget to delete an old version of a library or check in the same library twice without breaking the build now.

Our ClasspathCollisionCheckTask

I provide this task here, without any warranty. The source code is included in the JAR alongside the classes, if you want to know what it does exactly.

Assuming you already know how to use custom tasks within an ant build script, here’s only a short usage description.

Import the custom task:

<taskdef
    name="check.collision"
    classname="com.schneide.internal.anttask.ClasspathCollisionCheckTask"
    classpath="${customtasks.library.directory}/schneidetasks.jar"
/>

Next, use it on your classpath:

<check.collision verbose="true" failOnCollision="true">
    <path>
        <fileset dir="${classpath.library.directory}">
            <include name="**/*.jar"/>
        </fileset>
        <fileset dir="${internal.library.directory}">
            <include name="**/*.jar"/>
        </fileset>
    </path>
</check.collision>

The task scans the whole path you give it and reports any collision it detects. You will see the warnings in your build log.

If the failOnCollision parameter is set to true (optional, defaults to false), the build will abort after a collision. If you want to have debug information, set the verbose parameter to true (optional, defaults to false).

Conclusion

If you manage your project dependencies manually, you might find our custom ant task useful. If you use maven or ant ivy, you already have this functionality in your build process.

Feedback

I’m very interested in hearing your opinion on the task or about your way of handling dependencies. Leave us a comment.

Industry Standard C++

The other day I was browsing through the C++ API code of a third-party library. I was not much surprised to see stuff like

#define MAX(a, b) ( (a) >= (b) ? (a) : (b))
#define MIN(a, b) ( (a) <= (b) ? (a) : (b))

because despite the fact that std::min, std::max together with the rest of the C++ standard library is around for quite a while now, you still come across old fashioned code like above frequently. But things got worse:

#define FALSE 0
#define TRUE 1

and later:

...
bool someVariable = TRUE;

As if they learned only half the story about the bool type in C++. But there was more to come:

class ListItem
{
   ListItem* next;
   ListItem* previous;
   ...
};

class List : private ListItem
{
...
};

Yes, that’s right, the API guys created their own linked-list implementation. And a pretty weird one, too, mixing templates with void* pointers to hold the contents. Now, why on earth would you do that when you could just use std::list or std::vector? Makes you wonder about the quality of the rest of the code. Especially with C++ where there are so many little pitfalls and details which can burn you. Hey, if you have no clue about the very basics of a language, leave it alone!

Unfortunately, the above example is not exceptional in industry software. It seems that the C++ world these days is actually split into two worlds. In one, people like Andrei Alexandrescu write great books about Modern C++ design, Scott Meyers gives talks about Effective C++ and the boost guys introduce the next library using even more creative operator overloading that in the spirit library (which is pretty cool stuff, btw).

In the other world, you could easily call it industry reality, people barely know the STL, don’t use templates at all, or fall for misleading and dangerous c++ features like the throw() clause in method signatures. Or they ban certain c++ features because they are supposedly not easy to understand for the new guy on the project or are less readable in general. Take for example the Google C++ Style Guide. They don’t even allow exceptions, or the use of std::auto_ptr. Their take on the boost library is that “some of the libraries encourage … an excessively “functional” style of programming”. What exactly is bad about piece of functional programming used as the right tool in the right place? And what communicates ownership issues better than e.g. returning a heap allocated object using a std::auto_ptr?

The no-exceptions rule is also only partly understandable. Sure enough, exceptions increase code complexity in C++ more than in other languages (read Items 18 and 19 of Herb Sutter’s Exceptional C++ as an eye-opener. Or look here). But IMHO their advantages still outweigh their downsides.

With the upcoming new C++0X standard my guess is that the situation will not get any better, to put it mildly. Most likely, things like type inference with the new auto keyword will sell big because they save typing effort. Same thing with the long overdue feature of constructor delegation. But why would people who find functional programming less readable start to use lambda functions? As little known as the explicit keyword is now, how many people will know about or actually use the new “= delete” keyword, let alone “= default“? Maybe I’m a little too pessimistic here but I will certainly put a mark in my calender on the day I encounter the first concept definition in some piece of industry C++ software.

Update: Concepts have been removed from C++0X so that mark in my calender will not come any time soon…

Creative Wordle usage

Wordle used to summarize conference programs and analyze java projects.

Many of you may have stumbled over Wordle like I did some time ago. It is a nice little tool (implemented as a Java applet) that creates nice word clouds out of some arbitrary text pasted directly into the browser or provided by an URL. Since then I have found some nice, interesting and creative uses for it.

  • schneideblogwordleSchneide Blog Wordle First a Wordle-cloud of our blog frontpage. The left image shows that we are currently talking a lot about projects and our approach to blogging. Compare that with the cloud from a few weeks ago where listener structures and memory management were hot topics.
  • The guys over at EclipseSource ran Wordle over the EclipseCon program to give a quick overview what this conference is all about.
  • Daan analyzed the class naming of popular OpenSource projects and put the very interesting results on his blog.

I quickly hacked something similar together and ran it over two of our projects. The interesting thing is that you can actually get an impression what the projects are about. Let’s take a look at it:

NPA (Nano Particle Analyzer)

NPA Wordle
This seems to be a project where everything is about measuring something. We can see that there is need for calibration and that energy and data play a major role here. We can even spot a laser (try to find it!) out there which is an important part of the whole system.

Ramses

Ramses Wordle
This project seems to be very abstract and generic but there are some concrete pointers to what’s going on here too. It works together with some spectrometry hardware via Genie2k, with a Delphin box and some camera. There seems to be an appointment management integrated, i18n support ready and some more obscure things like fesas.

If you have other cool ideas how to use Wordle I would be glad to hear about them.

A DSL for deploying grails apps

Everytime I deploy my grails app I do the same steps over and over again:

  • get the latest build from our Hudson CI
  • extract the war file from the CI archive
  • scp the war to a gateway server
  • scp the war to the target server
  • run stop.sh to shutdown the jetty
  • run update.sh to update the web app in the jetty webapps dir
  • run start.sh to start the jetty

Reading the Productive Programmer I thought: “This should be automated”. Looking at the Rails world I found a tool named Capistrano which looked like a script library for deploying Rails apps. Using builders in groovy and JSch for SSH/scp I wrote a small script to do the tedious work using a self defined DSL for deploying grails apps:

Grapes grapes = new Grapes()
def script = grapes.script {
    set gateway: "gateway-server"
    set username: "schneide"
    set password: "************"
    set project: "my_ci_project"
    set ciType: "hudson"
    set target: "deploy_target.com"
    set ci_server: "hudson-schneide"
    set files: ["webapp.war"]

    task("deploy") {
        grab from: "ci"
        scp to: "target"
        ssh "stop.sh"
        ssh "update.sh"
        ssh "start.sh"
    }
}

script.tasks.deploy.execute()

This is far from being finished but a starting point and I think about open sourcing it. What do you think: may it help you? What are your experiences with deploying grails apps?

Make it visible: The Project Cockpit

How to use a whiteboard as information radiator for project management, showing progress, importance, urgency and volume of projects.

We are a project shop with numerous customers booking software development projects as they see fit, so we always work on several projects concurrently in various sub-teams.

We always strive for a working experience that provides more productivity and delight. One major concept of achieving it is “make it visible”. This idea is perfectly described in the awesome book “Behind Closed Doors” by Johanna Rothman and Esther Derby from the Pragmatic Bookshelf. Lets see how we applied the concept to the task of managing our project load.

What is the Project Cockpit?

The Project Cockpit is a whiteboard with titled index cards and separated regions. If you glance at it, you might be reminded of a scrum board. In effect, it serves the same purpose: Tracking progress (of whole projects) and making it visible.

Here is a photo of our Project Cockpit (with actual project names obscured for obvious reasons):

cockpit1

How does it work?

In summary, each project gets a card and transitions through its lifecycle, from left to right on the cockpit.

The Project Cockpit consists of two main areas, “upcoming projects” and “current projects”. Both areas are separated into three stages eachs, denoting the usual steps of project placing and project realization.

Every project we are contacted for gets represented by an index card with some adhesive tape and a whiteboard magnet on its back. The project card enters the cockpit on the left (in the “future” or “inquiry” region) and moves to the right during its lifecycle. The y-axis of the chart denotes the “importance” of the project, with higher being more important.

cockpit2

In the “upcoming” area, projects are in acquisition phase and might drop out to the bottom, either into the “delay filing” or the “trash”. The former is used if a project was blocked, but is likely to make progress in the future. The latter is the special place we put projects that went awry. It’s a seldom action, but finally putting a project card there was always a relief.

The more natural (and successful) progress of a project card is the advance from the “upcoming” area to the “present” bar. The project is now appointed and might get a redefinition on importance. Soon, it will enter the right area of “current” projects and be worked on.

The right area of “current” projects is a direct indicator of our current workload. From here on, project cards move to the rightmost bar labeled “past” projects. Past projects are achievements to be proud of (until the card magnet is needed for a new project card).

If you want to, you can color code the project cards for their urgency or apply fancy numbers stating their volume.

What’s the benefit?

The Project Cockpit enables every member of our company to stay informed about the project situation. It’s a great place to agree upon the importance of new projects and keep long running acquisitions (the delay filing cases) in mind. The whiteboard acts as an information radiator, everybody participates in project and workload planning because it’s always present. Unlike simpler approaches to the task, our Project Cockpit includes project importance, urgency and volume without overly complicating the matter.

The whiteboard occupies a wall in our meeting room, so every customer visiting us gets a glance on it. As we use internal code names, most customers even don’t spot their own project, let alone associate the other ones. But its always clear to them in which occupancy condition we are, without a word said about it.

Ultimately, we get visibility of very crucial information from our Project Cockpit: When the left side is crowded, it’s a pleasure, when the right side is crowded, it’s a pressure 😉

“Tag, you’re it!” – how we manage our blog heartbeat

We successfully revived a nearly abandonded blog by using a token and a few metaphors.

The new year 2009 just started. A great opportunity to review some things. Here is a review of our blogging.

heartbeatWe started this blog in February 2007. Soon afterwards, it was nearly dead, as no new articles were written. Why? We would have answered to “be under pressure” and “have more relevant things to do” or simply “have no idea about what to write”. The truth is that we didn’t regard this blog as being important to us. We didn’t allocate any ressources, be it time, topics or attention to it. Seeming unimportant is a sure death cause for any business resource in any mindful managed company.

The Revival

This changed in late August 2008, after we heard from several sources that our articles published so far were very promising. Some new contacts even asked about our Code Flow-O-Meter before they asked any other question. So we sat together and thought about a way to revive this blog with minimal possible effort. We came to the conclusion that, being a four-man-show company, it would be sufficient for everyone to write one blog article a month in order to show a weekly blog heartbeat. It’s simple math. The same discussion led to the conclusion that blogging in english would reach a broader audience.

It’s a management problem

This laid the foundation for a few new blog entries, as everyone was eager to tell some news. But how could we manage the blog heartbeat in a sustainable fashion, with minimal effort and attention of the individual?

blogtoken

We decided to give the “Blog Token” a try. This token is nothing more than a little index card informing you that you are responsible for the blog entry in the next week. You can keep the index card on your desk or take it with you to remind you of the task. If you published your entry, you hand it over to the following team member in line. The token order is defined on a very viewable whiteboard. It took us 5 minutes to set up the token and define the order. Everything else is managed by the one who wants to get rid of the token and the one who receives it.

It doesn’t work without metaphors

When we reviewed the process, we realized that without a few maxims and their impact, things would have gone astray even with the token in place. Here are some of our maxims, spelled by the metaphors we found for them:

  • “blog heartbeat”: When you want to “keep it flowing” in a sustainable pace, you need to have a pace first. We defined that our blog is alive when it has a periodic heartbeat. Weekly articles seemed to be a good start and were approved in every review yet.
  • “to grow vegetables”: Good ideas (and good blog topics) need to evolve and grow. You need to care about them for an amount of time and publish them when they are mature. But first of all, you need to put the seeds for ideas (and blog topics) in your garden. Whenever somebody mentions something that might be worth a blog entry, somebody calls “this is a vegetable!”. A first sketch of a new blog entry (a new vegetable in your garden) is born in this moment. To be honest, some vegetables starve over time.
  • “it’s not a competition”: We try to publish high quality blog entries. But it’s more important to us to tell you about our favorite vegetable (see second metaphor) than to win a pulitzer price for every article. We even try to remind ourselves that we do not compete for the recoginition from our readers (you, in this case!). To be honest here, too: Though it’s not a contest, we issued an internal price for our most read article: Using Hudson for C++/CMake/CppUnit

Reviewing the Revival

We revived our blog with three ingredients:

  • Our commitment (“it’s important to us”)
  • The Blog Token (“tag, you’re it!”)
  • Metaphors (“everyone can grow vegetables”)

Telling from the statistics, it simply skyrocketed us:

blog-stats

Thank YOU, our blog visitors, for making this possible. It’s been a great experience for us and we are looking forward to continue our blog heartbeat in 2009 with fully stacked vegetable gardens. Stay tuned and if you like, share your thoughts (or just say hello) by adding a comment. We really appreciate your opinion.