Make friends with your compiler

Suppose you are a C++ programmer on a project and you have the best intentions to write really good code. The one fellow that you better make friends with is your compiler. He will support you in your efforts whenever he can. Unless you don’t let him. One sure way to reject his help is to switch off all compiler warnings. I know it should be well-known by now that compiling at high warning levels is something to do always and anytime but it seems that many people just don’t do it.
Taking g++ as example, high warning levels do not mean just having “-Wall” switched on. Even if its name suggests otherwise, “-Wall” is just the minimum there. If you just spend like 5 minutes or so to look at the man page of g++ you find many many more helpful and reasonable -W… switches. For example (g++-4.3.2):


-Wctor-dtor-privacy: Warn when a class seems unusable because all the constructors or destructors in that class are private, and it has neither friends nor public static member functions.

Cool stuff! Let’s what else is there:


-Woverloaded-virtual: Warn when a function declaration hides virtual functions from a base class. Example:

class Base
{
public:
virtual void myFunction();
};

class Subclass : public Base
{
public:
void myFunction() const;
};

I would certainly like to be warned about that, but may be that’s just me.


-Weffc++: Warn about violations of the following style guidelines from Scott Meyers’ Effective C++ book

This is certainly one of the most “effective” weapons in your fight for bug-free software. It causes the compiler to spit out warnings about code issues that can lead to subtle and hard-to-find bugs but also about things that are considered good programming practice.

So suppose you read the g++ man page, you enabled all warning switches additional to “-Wall” that seem reasonable to you and you plan to compile your project cleanly without warnings. Unfortunately, chances are quite high that your effort will be instantly thwarted by third-party libraries that your code includes. Because even if your code is clean and shiny, header files of third-pary library may be not. Especially with “-Weffc++” this could result in a massive amount of warning messages in code that you have no control of. Even with the otherwise powerful, easy-to-use and supposedly mature Qt library you run into that problem. Compiling code that includes Qt headers like <QComboBox>with “-Weffc++” switched on is just unbearable.

Leaving aside the fact that my confidence in Qt has declined considerably since I noticed this, the question remains what to do ignore the shortcomings of other peoples code. With GCC you can for example add pragmas around offending includes as desribed here. Or you can create wrapper headers for third-party includes that contain


#pragma GCC system_header

AFAIK Microsoft’s compilers have similar pragmas:


#pragma warning(push, 1)
#include
#include
#pragma warning(pop)

Warning switches are a powerful tool to increase your code quality. You just have to use them!

Industry Standard C++

The other day I was browsing through the C++ API code of a third-party library. I was not much surprised to see stuff like

#define MAX(a, b) ( (a) >= (b) ? (a) : (b))
#define MIN(a, b) ( (a) <= (b) ? (a) : (b))

because despite the fact that std::min, std::max together with the rest of the C++ standard library is around for quite a while now, you still come across old fashioned code like above frequently. But things got worse:

#define FALSE 0
#define TRUE 1

and later:

...
bool someVariable = TRUE;

As if they learned only half the story about the bool type in C++. But there was more to come:

class ListItem
{
   ListItem* next;
   ListItem* previous;
   ...
};

class List : private ListItem
{
...
};

Yes, that’s right, the API guys created their own linked-list implementation. And a pretty weird one, too, mixing templates with void* pointers to hold the contents. Now, why on earth would you do that when you could just use std::list or std::vector? Makes you wonder about the quality of the rest of the code. Especially with C++ where there are so many little pitfalls and details which can burn you. Hey, if you have no clue about the very basics of a language, leave it alone!

Unfortunately, the above example is not exceptional in industry software. It seems that the C++ world these days is actually split into two worlds. In one, people like Andrei Alexandrescu write great books about Modern C++ design, Scott Meyers gives talks about Effective C++ and the boost guys introduce the next library using even more creative operator overloading that in the spirit library (which is pretty cool stuff, btw).

In the other world, you could easily call it industry reality, people barely know the STL, don’t use templates at all, or fall for misleading and dangerous c++ features like the throw() clause in method signatures. Or they ban certain c++ features because they are supposedly not easy to understand for the new guy on the project or are less readable in general. Take for example the Google C++ Style Guide. They don’t even allow exceptions, or the use of std::auto_ptr. Their take on the boost library is that “some of the libraries encourage … an excessively “functional” style of programming”. What exactly is bad about piece of functional programming used as the right tool in the right place? And what communicates ownership issues better than e.g. returning a heap allocated object using a std::auto_ptr?

The no-exceptions rule is also only partly understandable. Sure enough, exceptions increase code complexity in C++ more than in other languages (read Items 18 and 19 of Herb Sutter’s Exceptional C++ as an eye-opener. Or look here). But IMHO their advantages still outweigh their downsides.

With the upcoming new C++0X standard my guess is that the situation will not get any better, to put it mildly. Most likely, things like type inference with the new auto keyword will sell big because they save typing effort. Same thing with the long overdue feature of constructor delegation. But why would people who find functional programming less readable start to use lambda functions? As little known as the explicit keyword is now, how many people will know about or actually use the new “= delete” keyword, let alone “= default“? Maybe I’m a little too pessimistic here but I will certainly put a mark in my calender on the day I encounter the first concept definition in some piece of industry C++ software.

Update: Concepts have been removed from C++0X so that mark in my calender will not come any time soon…

Observer/Listener structures in C++ with boost’s smart pointers

Whenever you are developing sufficiently large complex programs in languages like C++ or Java you have to deal with memory issues. This holds true especially when your program is supposed to run 24/7 or close to that. Because these kinds of issues can be hard to get right Java has this nice little helper, the garbage collector. But as Java solves all memory problems, or maybe not? points out, you can still easily shoot yourself in foot or even blow your whole leg away.  One of the problems stated there is that memory leaks can easily occur due to incorrect listener relations. Whenever a listener is not removed properly, which is either a large object itself or has references to such objects,  it’s only a matter of time until your program dies with “OutOfMemoryError” as its last words.  One of the proposed solutions is to use Java weak pointers for listener management.  Let’s see how this translates to C++.

Observer/listener management in C++ is often done using pointers to listener objects. Pointers are pretty weak by default. They can be :

  • null
  • pointing to a valid object
  • pointing to an invalid memory address

In listener relationships especially the latter can be a problem. For example, simple listener management could look like this:

   class SimpleListenerManagement
   {
   public:
      void addListener(MyListener* listener);
      void removeListener(MyListener* listener);
      void notifyListeners();
   private:
      std::list<MyListener*> listeners_;
   };

   void SimpleListenerManagement::notifyListeners()
   {
      // call notify on all listeners
      for (std::list<MyListener*>::iterator iter = listeners_.begin();
          iter != listeners_.end();
          ++iter)
      {
         (*iter)->notify(); // may be a bad idea!
      }
   }

In notifyListeners(), the pointer is used trusting that it still points to a valid object. But if it doesn’t, for instance because the object was deleted but the client forgot to removed it from the listener management, well, too bad.

Obviously, the situation would be much better if we didn’t use raw pointers but some kind of wrapper objects instead.  A first improvement would be to use boost::shared_ptr in the listener management:

   typedef boost::shared_ptr<MyListener> MyListenerPtr;

   class SimpleListenerManagement
   {
   public:
      void addListener(MyListenerPtr listener);
      void removeListener(MyListenerPtr listener);
      void notifyListeners();
   private:
      std::list<MyListenerPtr> listeners_;
   };

Provided that the given MyListenerPtr instance was created correctly by the client we can be sure now that all listeners exist when we call notify() on them.  Seems much better now. But wait! Using boost::shared_ptr, we now hold  strong references in our listeners list and are therefore kind of in the same situation as described in the post mentioned above. If the client forgets to remove its MyListenerPtr instance it never gets deleted and may be in a invalid state next time notify() is called.

A solution that works well in most cases is to use boost::weak_ptr to hold the listeners. If you see boost::shared_ptr on a level with normal Java references, boost::weak_ptrs are roughly the same as Java’ s weak references. Our listener management class would then look like this:

   typedef boost::shared_ptr<MyListener> MyListenerPtr;
   typedef boost::weak_ptr<MyListener> MyListenerWeakPtr;

   class SimpleListenerManagement
   {
   public:
      void addListener(MyListenerPtr listener);
      void removeListener(MyListenerPtr listener);
      void notifyListeners();
   private:
      std::list<MyListenerWeakPtr> listeners_; // using weak_ptr
   };

Note that addListener and removeListener still use MyListenerPtr as parameter. This ensures that the client provides valid listener objects.  The interesting stuff happens in notifyListeners():

   void SimpleListenerManagement::notifyListeners()
   {
      std::list<MyListenerWeakPtr>::iterator iter = listeners_.begin();
      while(iter != listeners_.end())
      {
         if ((*iter).expired())
         {
            iter = listeners_.erase(iter);
         }
         else
         {
            MyListenerPtr listener = (*iter).lock(); // create a shared_ptr from the weak_ptr
            listener->notify();
            ++iter;
         }
      }
   }

Each weak_ptr can now be checked if its object still exists before using it. If the weak_ptr is expired, it can simply be removed from the listeners list. With this implementation the removeListener method becomes optional and can as well be omitted. The client only has to make sure that the shared_ptr holding the listener gets deleted somehow.

Award your Customer

Recently, we successfully finished a web app project that had many specialties we never had before. Major issues were very tight budget and time constraints (about 3 months) including an absolutely unpostponable deadline. However, the bigger concern for us was the diversity of our customer. Although we had one or two main reference persons, for the project to be successful we depended on the collaboration of a total of 8 departments.

As a first step to meet those challenges we decided on one-week iteration cycles – the shortest ever for us. At the kick-off meeting, where delegates of all departments were assembled, we presented our strategy and tried to make clear that communication and collaboration would be essential for the project to succeed. We also invited everyone to come to iteration meetings even when the agenda is not exactly about their specific requirements. After the meeting we hoped for the best.

With (almost) all departments it went like this: We did one requirements gathering appointment with one or two delegates and they either showed up once or twice on following meetings or they approved our implementation based on emailed screen shots. With most departments, email response time was good, with some, well, let’s just say holiday season didn’t really help. But altogether it was sufficient to keep the project well on track.

But wait! Did I say all departments? Not exactly! One single department actually managed it to sent at least one delegate to every single iteration meeting. And they not only enjoyed coffee and cookies but contributed a great deal every time. This was very helpful for us especially because after every iteration, we were a little bit more confident that we were still on the right track. Towards the end of the project, when success was foreseeable, we had the idea that their outstanding performance had to be rewarded somehow. So at the last iteration meeting, again with people from every department, we presented them with the Continuous Collaboration Award. ccaward They were very delighted and for the others it was a good laugh. And with the help of a little champagne and some snacks it became a very nice last iteration meeting.

As many of you know, good understanding between customer and developer can never be taken for granted. This is why agile methods always put great emphasis on extensive customer communication. A-Story-of-Project-Failure-Mitch-Lacey shows that even agile-by-the-book projects can fail basically due to lack of understanding on customer side. So do it like us and, if they deserve it, show your appreciation to your customer once in a while in a more creative way. And if you use a cup, make sure that there is also champagne around to fill it.

Using Hudson for C++/CMake/CppUnit

Update: Hudson for C++/CMake/CppUnit Revised

As a follow-up to Using grails projects in Hudson, here is another not-so-standard usage of Hudson: C++ projects with CMake and CppUnit. Let’s see how that works out.

As long as you have Java/Ant/JUnit based projects, a fine tool that it is, configuration of Hudson is pretty straight forward. But if you have a C++ project with CMake as build system and CppUnit for your unit testing, you have to dig a little deeper. Fortunately, Hudson provides the possibility to execute arbitrary shell commands. So in order to build the project and execute the tests, we can simply put a shell script to work:

   # define build and installation directories
   BUILD_DIR=$WORKSPACE/build_dir
   INSTALL_DIR=$WORKSPACE/install_dir

   # we want to have a clean build
   rm -Rf $BUILD_DIR
   mkdir $BUILD_DIR
   cd $BUILD_DIR

   # initializing the build system
   cmake  ..  -DCMAKE_INSTALL_PREFIX=$INSTALL_DIR

   # fire-up the compiler
   make install

Environment variable WORKSPACE is defined by Hudson. Other useful variables are e.g. BUILD_NUMBER, BUILD_TAG and CVS_BRANCH.

But what about those unit tests? Hudson understands JUnit test result files out-of-the-box. So all we have to do is make CppUnit spit out an xml report and then translate it to JUnit form. To help us with that, we need a little xslt transformation. But first, let’s see how we can make CppUnit generate xml results (a little simplified):

#include <cppunit/necessary/CppUnitIncludes/>
...

using namespace std;
using namespace CppUnit;

int main(int argc, char** argv)
{
   TestResult    controller;
   TestResultCollector result;
   controller.addListener(&result);

   CppUnit::TextUi::TestRunner runner;
   runner.addTest( TestFactoryRegistry::getRegistry().makeTest() );
   runner.run(controller);

   // important stuff happens next
   ofstream xmlFileOut("cpptestresults.xml");
   XmlOutputter xmlOut(&result, xmlFileOut);
   xmlOut.write();
}

The assumption here is that your unit tests are built into libraries that are linked with the main function above. To execute the unit tests we add the following to out shell script:

   export PATH=$INSTALL_DIR/bin:$PATH
   export LD_LIBRARY_PATH=$INSTALL_DIR/lib:$LD_LIBRARY_PATH

   # call the cppunit executable
   cd $WORKSPACE
   cppunittests

This results in CppUnit generating file $WORKSPACE/cpptestresults.xml. Now, with the help of a little program called xsltproc and the following little piece of XSLT code, we can translate cpptestresults.xml to testresults.xml in JUnit format.

 <?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
    <xsl:output method="xml" indent="yes"/>
    <xsl:template match="/">
        <testsuite>
            <xsl:attribute name="errors"><xsl:value-of select="TestRun/Statistics/Errors"/></xsl:attribute>
            <xsl:attribute name="failures">
                <xsl:value-of select="TestRun/Statistics/Failures"/>
            </xsl:attribute>
            <xsl:attribute name="tests">
                <xsl:value-of select="TestRun/Statistics/Tests"/>
            </xsl:attribute>
            <xsl:attribute name="name">from cppunit</xsl:attribute>
            <xsl:apply-templates/>
        </testsuite>
    </xsl:template>
    <xsl:template match="/TestRun/SuccessfulTests/Test">
        <testcase>
            <xsl:attribute name="classname" ><xsl:value-of select="substring-before(Name, '::')"/></xsl:attribute>
            <xsl:attribute name="name"><xsl:value-of select="substring-after(Name, '::')"/></xsl:attribute>
        </testcase>
    </xsl:template>
    <xsl:template match="/TestRun/FailedTests/FailedTest">
        <testcase>
            <xsl:attribute name="classname" ><xsl:value-of select="substring-before(Name, '::')"/></xsl:attribute>
            <xsl:attribute name="name"><xsl:value-of select="substring-after(Name, '::')"/></xsl:attribute>
            <error>
                <xsl:attribute name="message">
                    <xsl:value-of select=" normalize-space(Message)"/>
                </xsl:attribute>
                <xsl:attribute name="type">
                    <xsl:value-of select="FailureType"/>
                </xsl:attribute>
                <xsl:value-of select="Message"/>
                File:<xsl:value-of select="Location/File"/>
                Line:<xsl:value-of select="Location/Line"/>
            </error>
        </testcase>
    </xsl:template>
    <xsl:template match="text()|@*"/>
</xsl:stylesheet>

The following call goes into our shell script:

xsltproc cppunit2junit.xsl $WORKSPACE/cpptestresults.xml > $WORKSPACE/testresults.xml

In the configuration page we can now check “Display JUnit test results” and give testresults.xml as result file. As a last step, we can package everything in $WORKSPACE/install_dir into a .tgz file and have Hudson to store it as build artifact. That’s it!

As always, there is room for improvements. One would be to wrap the shell script code above in a separate bash script and have Hudson simply call that script. The only advantage of the approach above is that you can see what’s going on directly on the configuration page. If your project is bigger, you might have more than one CppUnit executable. In this case, you can for example generate all testresult.xml files into a separate directory and tell Hudson to take into account all .xml files there.

Update: For the CMake related part of the above shell script I recently published the first version of a cmakebuilder plugin for Hudson. Check out my corresponding blog post.