Are programming books overrated?

A little insight gathered through feedback from an internship. Software development books are somewhat overrated as they can’t teach practice well.

In the last few weeks, we had an internship of a student that just finished academic high school (“Gymnasium”) and is looking forward to take up studies in computer science. He wanted to get in touch with the practical aspects of the career he is about to choose. The programming courses in school merely covered the basics of a programming language (Java) and some UML.

We prepared the student for the internship by feeding him several books we thought were appropriate for his level of knowledge. The books were a beginner’s book about Java (Head First Java), an introduction to unit testing (Pragmatic Unit Testing) and a foundation on clean code programming (Refactoring). Our student read them thoroughly and could make references to the chapters during pair programming sessions.

Retrospective on the books

But one feedback we got from him was that the books alone were nearly useless for his case. If there wouldn’t have been tutorial style pair programming coding sessions and several short lectures , he couldn’t grasp the deeper meaning of the book chapters he read (he suffered from the “blank slate blockade” several times). This came a bit as a surprise for us, as the student was very clever and really into it. It wasn’t the student, it was the books.

But you can’t blame it on “Refactoring”, for example, as this book is an all-time classic filled with really important knowledge. It has to be the medium itself, books are not the ideal source to learn about programming and software development.

Books are part of the academics

There is an old question in our profession. It revolves around if we are more like engineers or artists, craftsmen or scientists. In the core of this question is a uncertainty about the right model of education. Artists and craftsmen prefer more practical training, with apprentice/master relations and personal knowledge transfer. For engineers and scientists, literature and more standardized lectures are better suited. Academic knowledge is transferred during debate, not during exercises.

The duality of our profession

Projecting the feedback of our student onto this question, there seems to be a duality in our profession: Both (or all four if you want) approaches are needed to form a whole. You can’t learn the theory and expect to excel on the job. But pratical experience alone will not suffice to keep up with the pace of our profession. Good books are like afterburners here, you’ll be hurled forward by every page.

Conclusion

If it’s really true that we need to learn our profession both ways at once, pair programming (in the tour guide or backseat driver style) is an essential part of our qualification. And our current university curriculum fails to deliver this part. Students nowadays can team up to program together on an assignment, but that’s not learning from a master (unless one in the team has distinctly more experience than everybody else and is able to transfer it). So I vote to bring more craftsmanship to the academic education, as the books alone won’t cut it.

Your opinion?

What’s your opinion on this topic? Drop us a line about your thoughts.

An Oracle story: Null, empty or what?

One big argument for relational databases is SQL which as a standard minimizes the effort needed to switch your app between different DBMSes. This comes particularily handy when using in-memory databases (like HSQL or H2) for development and a “big” one (like PostgreSQL, MySQL, DB2, MS SqlServer or Oracle) in production. The pity is that there are subtle differences with regard to the interpretation of the SQL-standard when it comes to databases from different vendors.

Oracle is particularily picky and offers quite some interesting behaviours: Most databases (all that I know well) treat null and empty as different values when it comes to strings. So it is perfectly valid to store an empty string in a not-null column and retrieving the string from the column yields an empty string. Not so with Oracle 10g! Inserting null and retrieving the value yields unsurprisingly null, even using Oracle. Inserting an empty string and retrieving the value leaves you with null, too! Oracle does not differentiate between empty strings and null values like a Java developer would expect. In our environment this has led to surprised developers and locally unreproducible bug which clearly exist in production a couple of times.

[rant]Oracle has great features for big installations and enterprises that can afford the support, maintenance and hardware of a serious Oracle DBMS installation. But IMHO it is a shame that such a big player in the market does not really care about the shortcomings of their flagship product and standards in general (Oracle 10g only supports SQL92 entry level!). Oracle, please fix such issues and help us developers to get rid of special casing for your database product![/rant]

The lesson to be learnt here is that you need a clone of the production database for your integration tests or acceptance tests to be really effective. Quite some bugs have slipped into production because of subtle differences in behaviour of our inhouse databases and the ones in production at the customer site.

Grails pitfall: Proxies and inheritance

When using inheritance in a lazy fetched association, proxies are a common pitfall in Grails.

Problem

You have a domain class hierarchy and want to use the classes in an association. You typically use the base class in a hasMany:

class Domain {
  static hasMany = [others:BaseClass]
}

Dependent on the actual class in the set others you want to call a different method:

public doSomething(domain) {
  domain.others.each {doThis(it)}
}

public doThis(ChildClassOne c) {}

public doThis(ChildClassTwo c) {}
...

Here the proxy mechanism of Hibernate (and Grails) causes a MethodMissingException to be thrown because the proxies are instances of the base class and not the actual child ones.
One way would be to activate eager fetching or you could use…

Solution: A dynamic visitor

Declare a visit method in the base class which takes a closure

def visit(Closure c) {
return c(this)
}

and make the dispatch in a method for the base class:

public doThis(BaseClass c) {
  return c.visit {doThis(it)}
}

Add flair to your code: Code Squiggles

Introduction to Code Squiggles, a coding style that improves the readability of your java code.

Wrought Iron by quadriremeFor several months now, I’m experimenting with a programming style that you might want to call “syntax aware programming”. Every coding step starts with the question “what do I want to achieve and how do I want to write it down?”. Then I proceed to write it down in this perfect manner, mostly some english sentence, and try to incrementally convert the syntax into something the java compiler stops complaining about. There’s a lot to be learnt about API design, naming and creative syntax usage this way. One thing I’ve discovered along the way are Code Squiggles.

Introduction to Code Squiggles

Code squiggles are little methods that contain no functionality, other than directly returning the single given parameter. Their purpose is to make the code more fluently readable by casual readers. Calling these methods is absolutely optional, as they offer no behaviour at runtime. But by bloating your code with these method calls, you lower the amount of syntax rules and conventions the reader has to know before being able to understand the code. The process of adding the method calls feels like adding “happy little squiggles” (I certainly miss Bob Ross!) to your code.

Adding Code Squiggles by example

Let me give you a full example how Code Squiggles can turn your boring old java code into something even non-programming readers can grasp without problems. Note that the process of transforming the code isn’t the process I initially described, but the way to deal with existing code.

The initial code fragment looks like this:

assertTrue(filesDirectory.getChild("0.png").isFile());

As you can see, it’s an assertion line of an unit test. We improve the readability by extracting the intent of the assertion into the called method name (it’s a normal “extract method” refactoring):

assertFileExists("0.png", filesDirectory);

Now is the time to add the first Code Squiggle:

assertFileExists("0.png", in(filesDirectory));

You’ve noticed the difference? It’s just the word “in”, but it improves the semantics of the second parameter. As I promised, Code Squiggles are methods without any functionality worth talking about:

protected VirtualFile in(VirtualFile filesDirectory) {
    return filesDirectory;
}

The method takes a parameter and returns it. The real “functionality” of this method lies within its name. Everything else is only necessary to overcome (or overcode) java’s shortcoming of advanced syntax definition measures.

That’s all about Code Squiggles. But it doesn’t stop here. Let’s improve the example to its final shape, when we add two squiggles:

assertFile(withName("0.png"), existsIn(filesDirectory));

Read this line out loud! And notice the subtle change in the assertion method’s name. It begins to deminish clarity without the Code Squiggles. That’s when your API begins to depend on them. But it’s still perfectly valid java code if you just omit them.

The conceptual origin of Code Squiggles clearly comes from the behaviour driven development (BDD) style of writing tests and the ScalaTest Matchers. So I can’t claim much originality or cleverness myself here. But Code Squiggles, despite this initial example, aren’t limited to test code.

Adding inline documentation with Code Squiggles

Remember the last time you needed to integrate an horrible third-party API? Probably, there was a method with seven or eight parameters and all of them were primitive ints. No matter how often you called this method, you couldn’t remember the order of these ints. That’s when Code Squiggles might help you a bit.

This is the signature of a fairly complex method:

protected int performCalculation(int value, int lowerLimit, int upperLimit, int offset) { ...

Therefore, your call isn’t very self-explanatory:

int result = performCalculation(10, 2, 23, 13);

But it might be a lot more understandable when you add some squiggles:

int result = performCalculation(value(10), lowerLimit(2), upperLimit(23), offset(13));

I won’t spell out the squiggle implementations, as they should be straight-forward. Note that the java compiler doesn’t catch up here. You can easily swap the squiggles around to obfuscate your code. All you got is read-time safety. If you want compile-time safety, you need to replace the primitives with types and probably pay for some rounds of beer for the third-party API developers to include your changes or write an adapter (“corruption layer” is my preferred term here).

Regular use of Code Squiggles

If you want to use squiggles a lot, you might think about collecting them in a dedicated class. Design them to be static and generic, and you can use them easily with static imports:

public static <T> T existsIn(T instance) {
    return instance;
}

But remember that there are reserved keywords in java that might hamper you a bit.

Conclusion

Code Squiggles are useless bloat to your code unless you value read-time safety or casual readability. They can tidy up your code to a point where method or even class names are affected to form a block of code where you only have to replace the funnier chars with blanks to obtain a paragraph of plain written english anybody can read and understand.

What are your ideas towards Code Squiggles? Have you used them on your code? Mind to share the result?

Code squiggles are little methods that contain no functionality, other than directly returning the single given parameter. Their purpose is to make the code more fluently readable by casual readers.

The C++ Shoot-yourself-in-the-Foot of the Week

I think we can all agree that C++, compared to other languages, provides quite a number of possibilities to screw up. Everybody working with the language at some point probably had problems with e.g. its exception system, operator overloading or automatic type conversions – to name just a few of the darker corners.

There are also many mitigation strategies which all come down to ban certain aspects of the language and/or define strict code conventions. If you follow Google’s style guide, for example, you cannot use exceptions and you are restricted to a very small set of boost libs.

But developers – being humans – often find creative and surprising ways to thwart every good intentions. In an external project the following conventions are in place:

  • Use const wherever you possibly can
  • Use boost::shared_ptr wherever it makes sense.
  • Define typedefs to shared_ptrs  in order to make code more readable.
  • typedefs to shared_ptrs are to be defined like this:
typedef boost::shared_ptr&lt;MySuperDuperClass&gt; MySuperDuperClassPtr;
  • typedefs to shared const pointers are to be defined like this:
typedef boost::shared_ptr&lt;const MySuperDuperClass&gt; MySuperDuperClassCPtr;

As you can see, postfixes Ptr and CPtr are the markers for normal shared_ptrs and constant shared_ptrs.

Last week, a compile error about some const to non-const conversion made me nearly pull my hair out. The types of variables that were involved all had the CPtr postfix but still the code didn’t compile. After a little digging I found that one of the typedefs involved was like this:

typedef boost::shared_ptr&lt;  MySuperDuperClass&gt; MySuperDuperClassCPtr;

Somebody just deleted the const modifier in front of MySuperDuperClass but left the name with the CPtr untouched. And because non-const to const conversions are allowed this was not detected for a couple of weeks. Nice going!

Any suggestions for a decent style checker for c++? Thanks in advance 😉

There should be a stakeholder for simplicity

You have stakeholders for your product idea, you have stakeholders for your clients and their ideas, you have stakeholders for clean code, for quality, for object oriented programming, … I think we need also stakeholders for simplicity.

Usually in every project ideas are abundant. Your client has many ideas what features he wants. You and your coworkers have a rich background in the problem domain and the technologies you use so that solutions are not sparse. But often this experience and confidence leads to abandoning an important trait: simplicity.
Most of the time you are focused on getting good solutions for the problems that arise. From past experiences with clients who could not exactly explain what they really need (which is different from what they want most of the time) you tend to include a little extra flexibility in your system. Maybe you need this and that variation some time in the future but at this very moment it’s a guess at best. And often this guess costs you. You could argue that you are investing into your project. But how many times did this investment really pay off? And how many times did you have a hard time just because you did not want to make restrictions? At first it seems like a little work. But with the next feature you have to continue supporting your little ‘extra’. Over time it infuses your system like leaven does it with bread. In the end it is more work to make it simple than to keep it simple.
So with every project you approach there should be a stakeholder for simplicity. Someone who focusses on simple solutions. Sometimes you have to cut a bit away from the feature or you have to view the problem from a different angle. Some other time you have to dig deeper into the problem domain or you need real data from your users (which is always better than what you can make up in your mind). Finding simple solutions is work but it is much more work to support your over-engineered solutions.

Improved Version of CMake Builder for Hudson

Introducing version 1.5 of cmake builder plugin for Hudson.

Today I just want to give a small round-up of the improvements made on the cmake builder plugin since my last blog post. Back then, version 1.2 was released to support master/slave configurations. As of yesterday, we are at version 1.5 which contains the following improvements/bug-fixes:

  • Bug: The drop-down box for selecting the build type didn’t remember its value. This was fixed with a patch by Atte Timonen.
  • Improvement: Also included in Atte’s patch was the propagation of environment variables to the cmake command which now allows to do parameterized builds. A big thank’s to Atte!
  • Improvement: The install command gets only executed when install directory and install command is given. Before, the build was either broken or $WORKSPACE was used automatically as install directory. Thanks to Dat Chu for his feedback.
  • Improvement: The one-line ‘Other CMake Arguments’ field can get full pretty quickly, so it was changed to a multi-line text-area.

Thank’s again for the feedback, and have fun with the new version!

Responsibility reduces waste

Most of the waste comes from being irresponsible. Over time waste becomes even harder to remove. So be responsible now.

Recently we participated in a local effort (site in German) to help making our environment cleaner by removing waste which was left by other people. When you take a look at your environment you may come to the conclusion that many people are irresponsible. Waste on the streets and in the parks, prohibition signs everywhere which name things and actions you wouldn’t even think of and when did people forget to flush the toilet?. And it doesn’t stop in the material world, you even find waste in your code. Allowing waste and not removing or preventing it leads to two effects:

  • even more waste (according to the broken window theory)
  • over time the waste becomes more and more intertwined with the environment

Imagine a plastic cup in a forest: When first thrown there it is clearly distinguishable from the mud, the leaves and its surroundings. Easy to see and easy to remove. But over time it is trampled over, crushed, hidden under leaves, wash over with mud, … in the end you may not even spot it when you look at the place where it was left.
The same happens to your code: You start with a small clearly defined part of bad smelly code and leave it alone. Now the first additional features come in, you add code, there and elsewhere. The surroundings change. You refactor. You move code. And in the end the once good known waste is littered all over your code and hard to remove.
So be responsible now! And don’t wait until the waste is hard or impossible to remove. Collective ownership (of code or of your environment) does not mean nobody is repsonsible, it means you are responsible.

Wrestling with Qt’s Model/View API – Filtering in Tree Models

Qt4’s model/view API can be kind of a challenge sometimes. Well, prepare for a even harder fight when sorting and filtering come into play.

As I described in one of my last posts, Qt4’s model/view API can be kind of a challenge sometimes. Well, prepare for a even harder fight when sorting and filtering come into play.

Let’s say you finally managed to get the data into your model and to provide correct implementations of the required methods in order for the attached views to display it properly. One of your next assignments after that is very likely something like implementing some kind of sorting and filtering of the model data. Qt provides a simple-at-first-sight proxy architecture for this with API class QSortFilterProxyModel as main ingredient.

Small preliminary side note: Last time I checked it was good OO practice to have only one responsibility for a given class. And wasn’t that even more important for good API design? Well, let’s not distract us with such minor details.

With my model implementation, none of the standard filtering mechanisms, like setting a filter regexp, were applicable, so I had to override method

QVariant filterAcceptsRow ( int source_row, const QModelIndex& sourceParent ) const

in order to make it work. Well, the rows disappeared as they should, but unfortunately so did all the columns except the first one. So what to do now? One small part of the documentation of QSortFilterProxyModel made me a little uneasy:

“… This simple proxy mechanism may need to be overridden for source models with more complex behavior; for example, if the source model provides a custom hasChildren() implementation you should also provide one in the proxy model.”

What on earth should I do with that? “… may need to be overridden“? “… for example.. hasChildren()…” Why can’t they just say clearly what methods must be overridden in which cases???

After a lot more trial and error I found that for whatever reason,

int columnCount ( const QModelIndex& parent ) const

had to be overridden in order for the columns to reappear. The implementation looks like what I had thought the proxy model would do already:

int MyFilter::columnCount ( const QModelIndex& parent ) const
{
   return sourceModel()->columnCount(parent);
}

So beware of QSortFilterProxyModel! It’s not as easy to use as it looks, and with that kind of fuzzy documentation it is even harder.

About PrintStream and Exceptions

Several of our projects deal with sensor hardware of different types often connected via the good old™ serial port. That is fine most of the time because most protocols are simple and RXTX provides a nice cross-platform library for most of your serial port needs. But many new computers do not feature the old RS232 serial ports anymore or other contraints prevent the use of a plain RS232 serial port. Here come serial converters like the Advantech ADAM 4570 (serial-to ethernet) or usb-to-serial converters into play. Usually this works fine.

Now one of our customers had a test system using an unreliable converter with sensor hardware. The hardware problems uncovered a robustness issue in our software which crashed the JVM when the virtual serial port of the converter disappeared and our app tried to write to it. Despite the faulty hardware our software had to be robust because it manages many more devices other than just that one sensor over serial. Looking at the problem we discovered that the crash occurred somewhere in the native part of RXTX. So we decided to scratch our own itch (and the one of the customer) and set out to fix the issue in RXTX at a Open Source Love Day (OSLD) . So we fixed the problem and submitted the patch to the bugtracker of the RXTX project. Our sample program now worked flawlessly and threw an IOException when the serial port failed in some way.

Happy to have fixed the problem we incorporated the patch RXTX in our production software but it still crashed and no IOException appeared anywhere in the logs. After another bughunting session we spotted the subtle difference of sample and production program: the use of OutputStream insted of PrintStream. PrintStream silently swallows all exceptions which proved fatal in our use case with the unreliable stream carrier. So the final fix was essentially replacing our PrintStream code

RXTXPort port = new RXTXPort("COM6");
PrintStream p = new PrintStream(port.getOutputStream(), true, "iso8859");
p.print("command");

with using OutputStream directly:

RXTXPort port = new RXTXPort("COM6");
OutputStream o = port.getOutputStream();
o.write("command".getBytes("iso8859"));

Conclusion

Be careful when using PrintStream with unreliable stream carriers it swallows exceptions! That may shadow problems which you may want to know of. Often PrintStreams behaviour will not be a problem but in certain cases like the one depicted above it causes a lot of headaches.