Category: Software Development

Combine cobertura with the awesomeness of crap4j

Want the awesomeness of crap4j without running your tests twice in your build? Just combine it with your cobertura data using crapertura.

You may have heard of crap4j when it was still actively developed. Crap4j is a software metric that points you to “crappy” methods in your projects by combining cyclomatic complexity numbers with test coverage data. The rationale is that overly complex code can only be tamed by rigorous testing or it will quickly reduce to an unmaintainable mess – the feared “rotten code” or “crappy code”, as Alberto Savoia and Bob Evans, the creators of crap4j would put it. The crap4j metric soon became our most important number for every project. It’s highly significant, yet easy to grasp and mandates a healthy coding style.

Some enhancements to crap4j

Crap4j got even better when we developed our own custom enhancements to it, like the CrapMap or the crap4j hudson plugin. We have a tool that formats the crap4j data like cobertura’s report, too.

A minor imperfection

The only thing that always bugged me when using crap4j inside our continuous integration build cycle was that at least half the data was already gathered. Cobertura calculates the code coverage of our tests right before crap4j does the same again. Wouldn’t it be great if the result of the first analysis could be re-used for the crap metric to save effort and time?

Different types of coverage

Soon, I learnt that crap4j uses the “path coverage” to combine it with the complexity of a method. This is perfectly reasonable given that the complexity determines the number of different pathes through the method. Cobertura only determines the “line coverage” and “branch coverage”. As it stands, you can’t use the cobertura data for crap4j because they represent different approaches to measure coverage. That’s still true and probably will be for a long time. But the allurement of the shortcut approach was too high for me to resist. I just tried it out one day to see the real difference.

A different metric

So, here it is, our new metric, heavily inspired by crap4j. I just took the line and branch coverage for every method and multiplied them. If you happen to have a perfect coverage (1.0 on both numbers), it stays perfect. If you only have 75% coverage on both numbers, it will result in a “crapertura coverage” of 56,25%. Then I fed this new coverage data into crap4j and compared the result with the original data. Well, it works on my project.

Presenting crapertura

Encouraged by this result, I wrote a complete ant task that acts similar to the original crap4j ant task. You can nearly use it as a drop-in replacement, given that the cobertura XML report file is already present. Here is an example ant call:


<crapertura
coberturaReportFile="/path/to/cobertura/coverage.xml"
targetDirectory="/where/to/place/the/crap4j/report"
classesDirectory="/your/unarchived/project/class/files"
/>

It will output the usual crap4j report files to the given target directory. Please note that even if it looks like crap4j data, it’s a different metric and should be treated as such. Therefore, online comparison of numbers is disabled.

The whole project is published on github. Feel free to browse the code and compile it for yourself. If you want a binary release, you might grab the latest jar from our download server.

The complete usage guide can be found on the github page or inside the project. If you have questions or issues, please use the comment section here.

Conclusion

If crapertura is able to give you nearly the numbers that crap4j gave you is up to your project, really. Our test project contained over 20k methods, but very little crap. The difference between crap4j and crapertura was negligible. Both metrics basically identified the same methods as being crappy. Your mileage may vary, though. If that’s the case, let us know. If your experience is like ours, you’ve just saved some time in your build cycle without sacrificing quality.

GORM Gotchas: Validation and hasMany

Using validation on the end of hasMany associations yields unexpected results.

The excellent GORM Gotchas Series inspired me to write about a gotcha I found recently.
You have a domain class Container which contains elements:

class Container {
  static hasMany = [elements:Element]

  List<Element> elements
}

and the element has a constraint:

class Element {
  static belongsTo = [container:Container]

  static constraints = {
    email(email: true)
  }

  String email
}

When you try to save a container with more than one element that fails validation, only the first error appears:

Container c = new Container()
c.addToElements(new Element(email: "a"))
c.addToElements(new Element(email: "b"))
c.save()
assertEquals(2, c.errors.allErrors.size()) // fails, only one error is recorded!

The solution described in the docs coined with In some situations (unusual situations)) is to use a custom validator in the container class:

class Container {

  static constraints = {
      elements(validator: { List val, Container obj ->
          val.each {Element element ->
            if(! element.validate()) {
              element.errors.allErrors.each { error->
                obj.errors.rejectValue(
                      'elements',
                      error.getCode(),
                      error.getArguments(),
                      error.getDefaultMessage()
                )
              }
            }
          }
          return true
      })
  }

  static hasMany = [elements:Element]

  List<Element> elements
}

Responsive Qt GUIs – Threading with Qt

Qt4 used to have only primitive threading support. Starting with version 4.4 new classes and functions makes your threading life a lot easier. So in case you haven’t come around to look at those features, do it now!, it’s worth it.

If you have used Qt4 for some time now, specifically since pre 4.4 versions, you may or may not aware of the latest developments in the threading part of the library. This post shall be a reminder in case you didn’t follow the versions in detail or just didn’t get around to look closer and/or update.

In pre 4.4 versions, the only way to do threading was to use class QThread. Subclass QThread, implement the run method, and there you had your thread. This looks fine at first, but, taking the title of the post as example, it can get annoying very fast. Sometimes you have just few lines of code you want to keep away from the GUI thread because, e.g. they could potentially block on some communication socket. Subclassing QThread for every small little work package is not something you want to do, so I guess many users just wrote their own thread pool or the like.

Starting with version4.4. Qt gained two major threading features, for which, IMHO, the Qt people do not a very good job of advertising. The first is QThreadPool together with QRunnable. All Java programmers, which use java.lang.Runnable since the beginning, may have their laugh now, I’ll wait…

The second new threading feature is the QtConcurrent namespace (from the Qt documentation):

The QtConcurrent namespace provides high-level APIs that make it possible to write multi-threaded programs without using low-level threading primitives such as mutexes, read-write locks, wait conditions, or semaphores

Sounds great! What else?

QtConcurrent includes functional programming style APIs for parallel list prosessing, including a MapReduce and FilterReduce implementation for shared-memory (non-distributed) systems, and classes for managing asynchronous computations in GUI applications.

This is really great stuff. Functions like QtConcurrent::run together with QFuture<T> and QFutureWatcher<T> can simplify your threading code significantly. So, if you haven’t got around to look at those new classes by now, I can only advise you to do it immediately. Allocate a refactoring slot in your next Sprint to replace all those old QThread sub-classes by shiny new QRunnables or QtConcurrent functions. It’s worth it!

Let’s get back to the responsive GUIs example. In his Qt Quarterly article, Witold Wysota describes in detail every technical possibility to keep your GUI responsive. It’s a very good article which provides a lot of insights. He starts with manual event processing and mentions the QtConcurrent features only at the very end of the article. I suggest the following order of threading-solutions-to-consider:

  1. QtConcurrent functions
  2. QThreadPool + QRunnable
  3. rest

Stay responsive!

Scala: Easier to read (and write), harder to understand?

There is a vivid discussion about Scala’s complexity going on for some weeks now on the web even with a response from Martin Odersky. I want to throw my 2¢ together with some hopefully new aspects into the discussion.
Scala’s liberal syntax rules and compiler magic like type inference and implicit conversions allow nicely written APIs and DSLs almost looking like prose texts. Take a look at APIs like scalatest and imagine the Java/Junit equivalent:

@Test def demonstrateScalaTest() {
  val sb = new StringBuilder("Scala")
  sb.append(" is cool!")
  sb.toString should be ("Scala is cool!")
  evaluating { "concise".charAt(-1) } should produce [StringIndexOutOfBoundsException]
}

There are really nice features that reduce day-to-day programming tasks to keywords or one-liners. Here are some examples:

// singletons have their own keyword (object), static does not exist!
object MySingleton {
  def printMessage {
    println("I am the only one")
 }
}

// lazy initialization/evaluation
lazy val complexResult = computeForHours()

// bean-style data container with two scala properties and one java-bean property with getter+setter
class Data(val readOnly: String, var readWrite: Int, @BeanProperty var javaProperty: String)

// tuples as return values or quick data transfer objects (DTO) for methods yielding multiple data objects
def correctCoords(x: Double, y: Double) = (x + 12, y * 0.5)
val (correctedX, correctedY) = correctCoords(0.37, 34.2)
println("corrected: " + correctedX + ", " + correctedY)

On the other hand there are so many features built-in that really make it hard to understand the code if you are not scala programmer with some experience. I like the differentiation between application and library code Martin Odersky himself makes in Programming Scala. The frameworks I have tried so far (Lift, scalatest and scala-swing) in Scala make your life very easy as long as you just use them. It is really a breeze and much more fun than using most APIs in Java for example. But when something goes wrong or you really want/have to understand what is going on you can have a hard time. This is true at least for a Scala beginner, sometimes perhaps for an pro, too.

Final Thoughts
In my opinion Scala is a very nice language that successfully combines clean object oriented programming with functional features. You can migrate from a pure OO-style to a nice hybrid “Scala-style” like many programmers did when they first used Java mostly with procedural style using classes only as namespaces for their static methods. I am quite sure that a Scala code style and best practices still have to develop. Programmers will need their time diving into the language and using it for their benefit. I hope Scala prospers and gains attention in the industry because I personally think it is a nice step forward compared to Java (which turns more and more into a mess where you need profound knowledge to fight your problems).

Regarding the complexity, which certainly exists in Scala, I only want to raise some questions which may be answered sometime in the future:

  • Maybe the tooling is just not there (yet)?
  • Maybe you sometimes just don’t have to understand everything what’s happening underneath?
  • Maybe Scala makes debugging much more seldom but harder, when something does not work out?
  • Maybe the features and power of Scala are worth learning?
  • Maybe certain features will just be banned by the teams like sometimes in Java teams (think of switch-case, the ?-operator, Autoboxing e.g.)?

== isn’t equals, or is it?

Beware of the subtle differences of == and equals in Java and Groovy.

== and equals behave different in Java (and Groovy). You all know the subtle difference when it comes to comparing strings. equals is recommended in Java, == works in Groovy, too. So you could think that using equals is always the better option… think again!
Take a look at the following Groovy code:

  String test = "Test"
  assertTrue("Test" == test) // true!
  assertTrue("Test" == "${test}") // true!
  assertTrue("Test".equals("${test}")) // false?!

The same happens with numbers:

  assertTrue(1L == 1) // true!
  assertTrue(1L.equals(1)) // false?!

A look at the API description of equals shows the underlying cause (taken from the Integer class):

Compares this object to the specified object. The result is true if and only if the argument is not null and is an Integer object that contains the same int value as this object.

equals follows the contract or best practice (see Effective Java) that the compared objects must be of the same class. So comparing different types via equals always results in false. You can convert both arguments to the same type beforehand to ensure that you get the expected behavior: the comparison of values. So next time when you compare two values think of the types or convert both values to the same type.

Java Swing Layouting done right

A praise of the most developer-friendly Java Swing layout manager to date: DesignGridLayout.

Layout Managers were an huge benefit for Java Swing. They enabled software developers to program layout rather than to “drag and drop” it with some proprietary GUI builder. That’s nothing against a good GUI builder, but against the “source code” that gets generated as a result of using it. But after some time of playing and working with the layout managers given by Swing itself, we concluded that they weren’t up to the task. Since then, we were constantly on the lookout for new and better ways to tackle the layouting task.

A history of layout managers

Let’s reiterate our major path with different layout managers:

  • GridBagLayout – the most versatile layout manager included in the Java Swing core classes. It’s capable to handle virtually every layouting task, but the price is huge constraint setup code. Since the code bloats with even facile complexity in the dialog, it’s not maintainable once written. The advantages over GUI builders aren’t really present.
  • StringGridBagLayout – has the same power as GridBagLayout, but with much more concise constraint definitions. It uses a string based domain specific language that you have to learn. After a while, you begin to feel a clumsiness when inserting variables into the constraints.
  • TableLayout – was a new approach to layouting by applying a global grid to your panel. You define the grid by specifying row and column constraints. If you need special cell constraints afterwards, you can alter them, but it’s getting bloated again.
  • StringTableLayout – provided a string based domain specific language over the TableLayout. It had some nice additional features, but lacked versatility with dynamic GUIs.
  • FormLayout – was a great relief and a good companion for many full sized layouting tasks. By concentrating on a problem domain (form based layouts), it played out some advantages over general purpose layout managers. This layout is still in use here.
  • MigLayout – the bigger brother of all these layouts. MigLayout comes with several pages of cheat sheets and you’re soon lost without it. It combines the approaches of all layout managers listed (and many more) and blends them into a massively powerful and versatile product. If you learn this layout manager thoroughly, you’ll never have to look elsewhere. But the learning curve is steep and the complexity of your code scales with the complexity of the GUI (which isn’t a drawback).

All these layout managers added value to our GUIs and are in use until today, albeit seldom.

Keep it simple

Most of the time, your dialogs aren’t these super-fancy, highly dynamic full-page layouts every UI designer dreams about. If they are, pick one of the layout managers from the list and wade through the constraint setup. But let’s say you want to layout a rather plain dialog with some widgets, but you want to do it quick without sacrificing the looks. Here is a developer-friendly solution for this task: Use the DesignGridLayout manager.

Slick and easy layouts

The one thing that differentiates the DesignGridLayout from almost every other layout manager is that you use the layout manager instance itself (in a fluent interface style) to arrange the constraints of your grid. You do not add your widgets to the panel and hope for the layout manager to catch up with the layout, you add them to the layout manager (and hope for it to fill it into your panel, which it does nicely). Here is a little example of the API usage:

JPanel content = new JPanel();
DesignGridLayout layout = new DesignGridLayout(content);
JTextArea history = new JTextArea();
history.setRows(5);
JTextField message = new JTextField();
JButton sendNow = new JButton("Send");
layout.row().grid(new JLabel("History:")).add(new JScrollPane(history));
layout.row().grid(new JLabel("Message:")).add(message, 2).add(sendNow);
content.setLayout(layout);

If you are interested in the possibilities of the layout manager, you should read the usage introduction page of DesignGridLayout.

Developer-friendly approach

One big advantage of the fluent API when compared with the string based constraint definitions is the compiler and type system support. You can’t spell anything wrong and the code completion feature of your IDE guides you to the right method and parameter order. The other advantage is that you don’t need to mess with pixel sizes for spacing and such. It’s handled by the layout manager in the most comfortable manner.

And because an article about a layout manager isn’t of any worth without a picture, here’s one:

This is a frame with the panel we constructed in the example code above.

Get the basics right

Nowadays with all the fancy stuff around, with features over features, bells and whistles it is even more important to get the basics right.

Nowadays with all the fancy stuff around, with features over features, bells and whistles it is even more important to get the basics right. But what are the basics?
If you apply for a job the first basic would be to read the job posting carefully. Many corporations require you to use a special keyword or cite the reference in a certain way. This is an easy way to avoid that the email ends up in the spam folder and it shows that you can also see who really read the job posting. But many get this wrong. Why? For me that’s one of the basics. Another basic breaker is many or highly visible typos. Once in a while we get some unusual and fancy looking applications with typos in the job title or in headlines. Hmmm.. why bother with time-consuming layouts and colors and have typos all over the place?
This trend can be seen in many places. We have a new and modern door opener. The buttons are in white and pastel colors. Which ruins the contrast. When the light is dim, I cannot make out a difference between the one for opening the door and switching on the light in the hallway. Looking fancy but useless.
The IT business is also good in breaking the basics. In the last weeks some of the major IDEs or frameworks brought out versions which had regression in one of the most basic places: version control. Why didn’t they catch it before release?
Why are features nowadays more important than the basics?

Non-trivial Custom Data in QActions

If you want to implement dynamic context menus with non-trivial custom data in your QActions, the Qt4 documentation is not very helpful. The article describes some solutions to this task.

Sometimes I get very frustrated with the online Qt4 documentation. Sure, the API docs are massive but for many parts they provide only very basic information. Unfortunately, many Qt books, too, often stop exactly at the point where it gets interesting.

One example for this are context menus. The API docs just show you how menus in general are created and how they are connected to the application: Basically, all menus are instances of QMenu which are filled with instances of QAction. QActions are used as representation of any kind of action than can be triggered from the GUI.

The standard method to connect QActions to the GUI controlling code is to use one of their signals, e.g. triggered(). This signal can be connected to a slot of your own class where you can then execute the corresponding action. This works fine as long as you have a limited set of actions that you all know at coding time. For example, a menu in your tool bar which contains actions Undo/Redo/Cut/Copy/Paste can be created very easily.

But there are use cases where you do not know in advance how many actions there will be in your menus. For example, in an application that provides a GUI for composing a complex data structure you may want to provide the user assisting context menus for adding new data parts depending on what parts already exist. Suddenly, you have to connect many actions to one slot and then you somehow have to know which QAction the user actually clicked.

Btw, let’s all recall the Command Pattern for a moment… ok, now on to some solutions.

Method 0 – QAction::setData: The QAction class provides method setData(), which can be used to store custom data in a QAction instance using QVariant as data wrapper. If you then use QMenu’s triggered signal, which gives you a pointer to the QAction that was clicked, you can extract your data from the QAction. I find this a little bit ugly since you have to wrap your data into QVariant which can get messy, if you want to provide more than one data element

Method 1 – Enhancing QAction::triggered(): By sub-classing QAction you can provide your own triggered() signal which you can enhance with all parameters you need in your slot.

class MyAction : public QAction
{
  Q_OBJECT
  public:
    MyAction(QString someActionInfo)
      : someActionInfo_(someActionInfo)
    {
      connect(this, SIGNAL(triggered()),
              this, SLOT(onTriggered()));
    }
  signals:
    void triggered(QString someActionInfo);
  private slots:
    void onTriggered() {
      emit triggered(someActionInfo_);
    }
  private:
    QString someActionInfo_;
};

This is nice and easy but limited to what data types can be transported via signal/slot parameters.

Method 2 – QSignalMapper: From the Qt4 docs on QSignalMapper:

This class collects a set of parameterless signals, and re-emits them with integer, string or widget parameters corresponding to the object that sent the signal.

… which is basically the same as we did in method 1.

Method 3 – Separate domain specific action classes: By the time the context menu is created you add QActions to the menu using QMenu’s addAction methods. Then you create instances of separate Command-like classes (as in Command Pattern) and connect them with the QAction’s triggered() signal:

// Command-like custom action class. No GUI related stuff here!
class MySpecialAction : public QObject
{
  Q_OBJECT
  public:
    MySpecialAction(QObject* parent, &lt;all necessary parameters to execute&gt;);

  public slots:
    void execute();
  ...
};

// create context menu
QAction* specialAction =
  menu-&gt;addAction(&quot;Special Action Nr. 1&quot;);
MySpecialAction* mySpecialAction =
  new MySpecialAction(specialAction, ...);
connect(specialAction, SIGNAL(triggered()),
        mySpecialAction, SLOT(execute()));

As you can see, QAction specialAction is parent of mySpecialAction, thereby taking ownership of mySpecialAction. This is my preferred approach because it is the most flexible in terms of what custom data can be stored in the command. Furthermore, the part that contains the execution code – MySpecialAction – has nothing at all to do with GUI stuff and can easily be used in other parts of the system, e.g. non-GUI system interfaces.

How have you solved this problem?

Are programming books overrated?

A little insight gathered through feedback from an internship. Software development books are somewhat overrated as they can’t teach practice well.

In the last few weeks, we had an internship of a student that just finished academic high school (“Gymnasium”) and is looking forward to take up studies in computer science. He wanted to get in touch with the practical aspects of the career he is about to choose. The programming courses in school merely covered the basics of a programming language (Java) and some UML.

We prepared the student for the internship by feeding him several books we thought were appropriate for his level of knowledge. The books were a beginner’s book about Java (Head First Java), an introduction to unit testing (Pragmatic Unit Testing) and a foundation on clean code programming (Refactoring). Our student read them thoroughly and could make references to the chapters during pair programming sessions.

Retrospective on the books

But one feedback we got from him was that the books alone were nearly useless for his case. If there wouldn’t have been tutorial style pair programming coding sessions and several short lectures , he couldn’t grasp the deeper meaning of the book chapters he read (he suffered from the “blank slate blockade” several times). This came a bit as a surprise for us, as the student was very clever and really into it. It wasn’t the student, it was the books.

But you can’t blame it on “Refactoring”, for example, as this book is an all-time classic filled with really important knowledge. It has to be the medium itself, books are not the ideal source to learn about programming and software development.

Books are part of the academics

There is an old question in our profession. It revolves around if we are more like engineers or artists, craftsmen or scientists. In the core of this question is a uncertainty about the right model of education. Artists and craftsmen prefer more practical training, with apprentice/master relations and personal knowledge transfer. For engineers and scientists, literature and more standardized lectures are better suited. Academic knowledge is transferred during debate, not during exercises.

The duality of our profession

Projecting the feedback of our student onto this question, there seems to be a duality in our profession: Both (or all four if you want) approaches are needed to form a whole. You can’t learn the theory and expect to excel on the job. But pratical experience alone will not suffice to keep up with the pace of our profession. Good books are like afterburners here, you’ll be hurled forward by every page.

Conclusion

If it’s really true that we need to learn our profession both ways at once, pair programming (in the tour guide or backseat driver style) is an essential part of our qualification. And our current university curriculum fails to deliver this part. Students nowadays can team up to program together on an assignment, but that’s not learning from a master (unless one in the team has distinctly more experience than everybody else and is able to transfer it). So I vote to bring more craftsmanship to the academic education, as the books alone won’t cut it.

Your opinion?

What’s your opinion on this topic? Drop us a line about your thoughts.

An Oracle story: Null, empty or what?

One big argument for relational databases is SQL which as a standard minimizes the effort needed to switch your app between different DBMSes. This comes particularily handy when using in-memory databases (like HSQL or H2) for development and a “big” one (like PostgreSQL, MySQL, DB2, MS SqlServer or Oracle) in production. The pity is that there are subtle differences with regard to the interpretation of the SQL-standard when it comes to databases from different vendors.

Oracle is particularily picky and offers quite some interesting behaviours: Most databases (all that I know well) treat null and empty as different values when it comes to strings. So it is perfectly valid to store an empty string in a not-null column and retrieving the string from the column yields an empty string. Not so with Oracle 10g! Inserting null and retrieving the value yields unsurprisingly null, even using Oracle. Inserting an empty string and retrieving the value leaves you with null, too! Oracle does not differentiate between empty strings and null values like a Java developer would expect. In our environment this has led to surprised developers and locally unreproducible bug which clearly exist in production a couple of times.

[rant]Oracle has great features for big installations and enterprises that can afford the support, maintenance and hardware of a serious Oracle DBMS installation. But IMHO it is a shame that such a big player in the market does not really care about the shortcomings of their flagship product and standards in general (Oracle 10g only supports SQL92 entry level!). Oracle, please fix such issues and help us developers to get rid of special casing for your database product![/rant]

The lesson to be learnt here is that you need a clone of the production database for your integration tests or acceptance tests to be really effective. Quite some bugs have slipped into production because of subtle differences in behaviour of our inhouse databases and the ones in production at the customer site.