Category: Year

Developing Grails Apps – Some Dark Sides

Most of the time, developing Grails apps is a nice experience. But there are also dark sides. One of which is when bugs do appear or do not appear depending on how you started your app.

Usually, I try to avoid it but this time a Disclaimer is in order: This is not a Grails rant. Most of the time developing Grails projects is fast and smooth. Using Grails brought many advantages for us. But there are also dark sides…

My main criticism is that Grails abstractions are more than leaky! In every list of examples for the definition of the term Leaky Abstraction Grails should be top. As soon as you leave the tutorial/scaffolding/helloworld level you have to know a lot about the underlying stack. And with Hibernate and Spring neither of the words small, easy and lightweight do apply.

GORM, too, is only easy to use at first sight. The very informative blog series about GORM gotchas should absolutely become part of the user guide or the refence docs.

And there are those times where it gets really unpleasant. This is e.g. when a bug does appear in your grails application running in a servlet container (packaged in a .war)  but does not appear when the application is started from within the IDE. Our last one of those was a naming conflict in a .gsp file. The controller handed a model like this to the .gsp:

...
return [fieldValue: 'THE_VALUE', ...]

The model entry ‘fieldValue’ was used in the .gsp to set the value of a combo box. Unfortunately, ‘fieldValue’ is also the name of a built-in Grails tag …

Admittedly, ‘fieldValue’ was not the wisest choice of names and I would certainly expect to get scolded loudly by Grails for that – ideally with a nice descriptive exception. But what happend instead led to a loud scolding of Grails from us. And to some big question marks: What is the difference between executing Grails from the IDE and within a servlet container with respect to naming resolution? Why is there a difference, at all?

We had a hard time figuring out this one, not least because the error message was not very telling. And since this was not the first of those works-in-the-IDE-but-not-in-a-real-environment bugs there is always this slightly uneasy feeling…

As I said in the beginning, most of the time developing Grails applications is nice and shiny. I would not support their slogan, though. My personal search for the best web development tool is definitively not over.

How about your search?

Combine cobertura with the awesomeness of crap4j

Want the awesomeness of crap4j without running your tests twice in your build? Just combine it with your cobertura data using crapertura.

You may have heard of crap4j when it was still actively developed. Crap4j is a software metric that points you to “crappy” methods in your projects by combining cyclomatic complexity numbers with test coverage data. The rationale is that overly complex code can only be tamed by rigorous testing or it will quickly reduce to an unmaintainable mess – the feared “rotten code” or “crappy code”, as Alberto Savoia and Bob Evans, the creators of crap4j would put it. The crap4j metric soon became our most important number for every project. It’s highly significant, yet easy to grasp and mandates a healthy coding style.

Some enhancements to crap4j

Crap4j got even better when we developed our own custom enhancements to it, like the CrapMap or the crap4j hudson plugin. We have a tool that formats the crap4j data like cobertura’s report, too.

A minor imperfection

The only thing that always bugged me when using crap4j inside our continuous integration build cycle was that at least half the data was already gathered. Cobertura calculates the code coverage of our tests right before crap4j does the same again. Wouldn’t it be great if the result of the first analysis could be re-used for the crap metric to save effort and time?

Different types of coverage

Soon, I learnt that crap4j uses the “path coverage” to combine it with the complexity of a method. This is perfectly reasonable given that the complexity determines the number of different pathes through the method. Cobertura only determines the “line coverage” and “branch coverage”. As it stands, you can’t use the cobertura data for crap4j because they represent different approaches to measure coverage. That’s still true and probably will be for a long time. But the allurement of the shortcut approach was too high for me to resist. I just tried it out one day to see the real difference.

A different metric

So, here it is, our new metric, heavily inspired by crap4j. I just took the line and branch coverage for every method and multiplied them. If you happen to have a perfect coverage (1.0 on both numbers), it stays perfect. If you only have 75% coverage on both numbers, it will result in a “crapertura coverage” of 56,25%. Then I fed this new coverage data into crap4j and compared the result with the original data. Well, it works on my project.

Presenting crapertura

Encouraged by this result, I wrote a complete ant task that acts similar to the original crap4j ant task. You can nearly use it as a drop-in replacement, given that the cobertura XML report file is already present. Here is an example ant call:


<crapertura
coberturaReportFile="/path/to/cobertura/coverage.xml"
targetDirectory="/where/to/place/the/crap4j/report"
classesDirectory="/your/unarchived/project/class/files"
/>

It will output the usual crap4j report files to the given target directory. Please note that even if it looks like crap4j data, it’s a different metric and should be treated as such. Therefore, online comparison of numbers is disabled.

The whole project is published on github. Feel free to browse the code and compile it for yourself. If you want a binary release, you might grab the latest jar from our download server.

The complete usage guide can be found on the github page or inside the project. If you have questions or issues, please use the comment section here.

Conclusion

If crapertura is able to give you nearly the numbers that crap4j gave you is up to your project, really. Our test project contained over 20k methods, but very little crap. The difference between crap4j and crapertura was negligible. Both metrics basically identified the same methods as being crappy. Your mileage may vary, though. If that’s the case, let us know. If your experience is like ours, you’ve just saved some time in your build cycle without sacrificing quality.

Open Source Love Day September 2010

Our Open Source Love Day for September 2010 brought love for the cmake hudson plugin and a brand new tool around crap4j that will be released soon.

On wednesday last week, we held our Open Source Love Day for September 2010. Our day started with the usual Homepage Comittee meeting and very soon, we were up and working. This time, our success rate wasn’t as high as we wanted, mostly because we worked on internal tools that didn’t work out quite as well as expected. But, we managed to produce something valuable this day.

The Open Source Love Day

We introduced a monthly Open Source Love Day (OSLD) to show our appreciation to the Open Source software ecosystem and to donate back. We heavily rely on Open Source software for our projects. We would be honored if you find our contributions useful. Check out our first OSLD blog posting for details on the event itself.

On this OSLD, we accomplished the following tasks:

  • A new version 1.6.1 of the cmakebuilder hudson plugin was published. This version consisted of bugfixes only and right now, it still seems flawed. We are working on the issue, expect a new version 1.6.2 soon.
  • Our internal time-tracking tool got love at several issues. One issue required the use of triangle-shaping CSS, as described in this blog post from Jon Rohan. Our issue weren’t finished because Javascript code can rot into a big pile of crap really quick.
  • We managed to make a long hatched dream come true at this OSLD. As you might be aware, we are big fanboys of crap4j, a metric tool that associates test coverage with code complexity. Thus, we wrote the crap4j hudson plugin, release the CrapMap and use some internal improvements, too. The main disadvantage of crap4j is the strong dependency on a specific test coverage tool. Our goal was to use the test coverage data we already collect using Cobertura. We achieved this goal and got the whole thing working. It will be released in the next few weeks, with a detailed blog post here. Stay tuned for this new tool (it already has a name: Crapertura).

What were our lessons learnt today?

  • No matter how clever you are, Javascript outsmarts you every time it appears in superior numbers. Refactoring is the key here, but difficult and tedious to apply.
  • When you dissect a foreign API or code base, you just need to find the right grip. I cannot decribe it more precise right now, but this grip is all you need to open up the code. When playing around with the crap4j code base, as soon as we held the grip, everything else followed naturally. Perhaps “the grip” can be translated with “catching the author’s intent”. These are always magical moments.

Retrospective of the OSLD

This OSLD wasn’t as successful as we wished, partly because of missing manpower (honeymoon holiday!) and because of our inability to tame a Javascript code base. We have to work on our expertise here and we are glad that we’ve found out at an OSLD, not a time- and mission-critical customer project.

GORM Gotchas: Validation and hasMany

Using validation on the end of hasMany associations yields unexpected results.

The excellent GORM Gotchas Series inspired me to write about a gotcha I found recently.
You have a domain class Container which contains elements:

class Container {
  static hasMany = [elements:Element]

  List<Element> elements
}

and the element has a constraint:

class Element {
  static belongsTo = [container:Container]

  static constraints = {
    email(email: true)
  }

  String email
}

When you try to save a container with more than one element that fails validation, only the first error appears:

Container c = new Container()
c.addToElements(new Element(email: "a"))
c.addToElements(new Element(email: "b"))
c.save()
assertEquals(2, c.errors.allErrors.size()) // fails, only one error is recorded!

The solution described in the docs coined with In some situations (unusual situations)) is to use a custom validator in the container class:

class Container {

  static constraints = {
      elements(validator: { List val, Container obj ->
          val.each {Element element ->
            if(! element.validate()) {
              element.errors.allErrors.each { error->
                obj.errors.rejectValue(
                      'elements',
                      error.getCode(),
                      error.getArguments(),
                      error.getDefaultMessage()
                )
              }
            }
          }
          return true
      })
  }

  static hasMany = [elements:Element]

  List<Element> elements
}

Why I give lectures in software engineering

I’m often asked why I give lectures in software engineering, as they appear to not pay off for me. I think they do and here is why.

<a href="http://de.fotolia.com/id/21746212" mce_href="http://de.fotolia.com/id/21746212" title="" alt="">falcn</a> - Fotolia.com

For more than eight years, I give lectures in software engineering, object oriented programming and software development “best practices”. I have to spend nearly a day every week for six months in the year to prepare and hold the classes. My normal work schedule is always very stuffed with tasks, I have to affront my other duties sometimes in order to show up in front of the students. On many occassions, I’ve been asked why I keep giving lectures despite pressing liabilities, inferior payment and generally better alternatives elsewhere.

Here is a list with answers I’ve given to this questions over and over again. I do not want to convince you that giving lectures is the best thing right after sliced bread or that you will experience any of these if you manage to get in the same position. It’s just a rational explanation why the question still strikes me as odd.

  • It’s pure fun – This surely doesn’t count for everyone, but for me, speaking (ranting, raving, arguing) about software development counts as fun times. Being “on stage” in front of the students helps me to free my thoughts from dead freight and completely concentrate on the topics.
  • I’m being paid to recapitulate the basics – This are two advantages in one: being paid cannot be bad (albeit payment can always improve) and to repeat the basics of my craft on a regular schedule can be seen in the tradition of katas. I’m very bulletproof in discussions about fundamental topics of software engineering because I’ve heard most questions and had to answer them multiple times already.
  • I’m constantly learning new facets about well-known topics – My students always bring in unique and original thoughts about topics that I thought to have mastered. And then, a new way to access things emerges, at least for me. I feel very certain that I’m still learning more during the lectures than my students do. And feedback suggests that they learn a lot.
  • I’m honing my verbal abilities – Giving a lecture is all about speaking without script and responding to the audience. You have to make your points, but you cannot force them. Sometimes I feel like a stand-up comedian for technical knowledge. Having the ability to speak fluently while preparing the next topic in the back of your head is a great advantage in every situation including verbal communication.
  • Roughly 100 aspiring developers remember me every year – What they will remember me for can be debated about, but they will remember. This is all about “networking”, but focussed on members of my own profession. The reach of this network amazes me every time when it loops back.
  • I can contact every local company with job training – Due to the nature of the Cooperative State University where I’m giving my lectures, I can also establish contact to every software company in the vicinity. Many contacts would never happen without my function as lecturer.
  • I keep in touch with hypes – Students are easy prey for IT hypes. Their experience with different technologies isn’t embittered by analogies from the past. All I have to do is to listen to them when they tell me about their work and hobby projects. And then I can draw my own conclusions based on their first-hand experience.

All these reasons and some more are enough for me to stick with the job. You can see a lot of short term benefits and some aspects that might pay off at medium term. On the long run, I’m convinced that my personal advantages from this job will outweight the (sometimes serious) drawbacks. And then, I haven’t yet included the advantages that my students took along from my lectures, hopefully.

If you happen to give lectures too, I would be pleased if you blog about your reasons for doing so, and announce your post here. Or just use the comment section.

Responsive Qt GUIs – Threading with Qt

Qt4 used to have only primitive threading support. Starting with version 4.4 new classes and functions makes your threading life a lot easier. So in case you haven’t come around to look at those features, do it now!, it’s worth it.

If you have used Qt4 for some time now, specifically since pre 4.4 versions, you may or may not aware of the latest developments in the threading part of the library. This post shall be a reminder in case you didn’t follow the versions in detail or just didn’t get around to look closer and/or update.

In pre 4.4 versions, the only way to do threading was to use class QThread. Subclass QThread, implement the run method, and there you had your thread. This looks fine at first, but, taking the title of the post as example, it can get annoying very fast. Sometimes you have just few lines of code you want to keep away from the GUI thread because, e.g. they could potentially block on some communication socket. Subclassing QThread for every small little work package is not something you want to do, so I guess many users just wrote their own thread pool or the like.

Starting with version4.4. Qt gained two major threading features, for which, IMHO, the Qt people do not a very good job of advertising. The first is QThreadPool together with QRunnable. All Java programmers, which use java.lang.Runnable since the beginning, may have their laugh now, I’ll wait…

The second new threading feature is the QtConcurrent namespace (from the Qt documentation):

The QtConcurrent namespace provides high-level APIs that make it possible to write multi-threaded programs without using low-level threading primitives such as mutexes, read-write locks, wait conditions, or semaphores

Sounds great! What else?

QtConcurrent includes functional programming style APIs for parallel list prosessing, including a MapReduce and FilterReduce implementation for shared-memory (non-distributed) systems, and classes for managing asynchronous computations in GUI applications.

This is really great stuff. Functions like QtConcurrent::run together with QFuture<T> and QFutureWatcher<T> can simplify your threading code significantly. So, if you haven’t got around to look at those new classes by now, I can only advise you to do it immediately. Allocate a refactoring slot in your next Sprint to replace all those old QThread sub-classes by shiny new QRunnables or QtConcurrent functions. It’s worth it!

Let’s get back to the responsive GUIs example. In his Qt Quarterly article, Witold Wysota describes in detail every technical possibility to keep your GUI responsive. It’s a very good article which provides a lot of insights. He starts with manual event processing and mentions the QtConcurrent features only at the very end of the article. I suggest the following order of threading-solutions-to-consider:

  1. QtConcurrent functions
  2. QThreadPool + QRunnable
  3. rest

Stay responsive!

Scala: Easier to read (and write), harder to understand?

There is a vivid discussion about Scala’s complexity going on for some weeks now on the web even with a response from Martin Odersky. I want to throw my 2¢ together with some hopefully new aspects into the discussion.
Scala’s liberal syntax rules and compiler magic like type inference and implicit conversions allow nicely written APIs and DSLs almost looking like prose texts. Take a look at APIs like scalatest and imagine the Java/Junit equivalent:

@Test def demonstrateScalaTest() {
  val sb = new StringBuilder("Scala")
  sb.append(" is cool!")
  sb.toString should be ("Scala is cool!")
  evaluating { "concise".charAt(-1) } should produce [StringIndexOutOfBoundsException]
}

There are really nice features that reduce day-to-day programming tasks to keywords or one-liners. Here are some examples:

// singletons have their own keyword (object), static does not exist!
object MySingleton {
  def printMessage {
    println("I am the only one")
 }
}

// lazy initialization/evaluation
lazy val complexResult = computeForHours()

// bean-style data container with two scala properties and one java-bean property with getter+setter
class Data(val readOnly: String, var readWrite: Int, @BeanProperty var javaProperty: String)

// tuples as return values or quick data transfer objects (DTO) for methods yielding multiple data objects
def correctCoords(x: Double, y: Double) = (x + 12, y * 0.5)
val (correctedX, correctedY) = correctCoords(0.37, 34.2)
println("corrected: " + correctedX + ", " + correctedY)

On the other hand there are so many features built-in that really make it hard to understand the code if you are not scala programmer with some experience. I like the differentiation between application and library code Martin Odersky himself makes in Programming Scala. The frameworks I have tried so far (Lift, scalatest and scala-swing) in Scala make your life very easy as long as you just use them. It is really a breeze and much more fun than using most APIs in Java for example. But when something goes wrong or you really want/have to understand what is going on you can have a hard time. This is true at least for a Scala beginner, sometimes perhaps for an pro, too.

Final Thoughts
In my opinion Scala is a very nice language that successfully combines clean object oriented programming with functional features. You can migrate from a pure OO-style to a nice hybrid “Scala-style” like many programmers did when they first used Java mostly with procedural style using classes only as namespaces for their static methods. I am quite sure that a Scala code style and best practices still have to develop. Programmers will need their time diving into the language and using it for their benefit. I hope Scala prospers and gains attention in the industry because I personally think it is a nice step forward compared to Java (which turns more and more into a mess where you need profound knowledge to fight your problems).

Regarding the complexity, which certainly exists in Scala, I only want to raise some questions which may be answered sometime in the future:

  • Maybe the tooling is just not there (yet)?
  • Maybe you sometimes just don’t have to understand everything what’s happening underneath?
  • Maybe Scala makes debugging much more seldom but harder, when something does not work out?
  • Maybe the features and power of Scala are worth learning?
  • Maybe certain features will just be banned by the teams like sometimes in Java teams (think of switch-case, the ?-operator, Autoboxing e.g.)?

== isn’t equals, or is it?

Beware of the subtle differences of == and equals in Java and Groovy.

== and equals behave different in Java (and Groovy). You all know the subtle difference when it comes to comparing strings. equals is recommended in Java, == works in Groovy, too. So you could think that using equals is always the better option… think again!
Take a look at the following Groovy code:

  String test = "Test"
  assertTrue("Test" == test) // true!
  assertTrue("Test" == "${test}") // true!
  assertTrue("Test".equals("${test}")) // false?!

The same happens with numbers:

  assertTrue(1L == 1) // true!
  assertTrue(1L.equals(1)) // false?!

A look at the API description of equals shows the underlying cause (taken from the Integer class):

Compares this object to the specified object. The result is true if and only if the argument is not null and is an Integer object that contains the same int value as this object.

equals follows the contract or best practice (see Effective Java) that the compared objects must be of the same class. So comparing different types via equals always results in false. You can convert both arguments to the same type beforehand to ensure that you get the expected behavior: the comparison of values. So next time when you compare two values think of the types or convert both values to the same type.

Java Swing Layouting done right

A praise of the most developer-friendly Java Swing layout manager to date: DesignGridLayout.

Layout Managers were an huge benefit for Java Swing. They enabled software developers to program layout rather than to “drag and drop” it with some proprietary GUI builder. That’s nothing against a good GUI builder, but against the “source code” that gets generated as a result of using it. But after some time of playing and working with the layout managers given by Swing itself, we concluded that they weren’t up to the task. Since then, we were constantly on the lookout for new and better ways to tackle the layouting task.

A history of layout managers

Let’s reiterate our major path with different layout managers:

  • GridBagLayout – the most versatile layout manager included in the Java Swing core classes. It’s capable to handle virtually every layouting task, but the price is huge constraint setup code. Since the code bloats with even facile complexity in the dialog, it’s not maintainable once written. The advantages over GUI builders aren’t really present.
  • StringGridBagLayout – has the same power as GridBagLayout, but with much more concise constraint definitions. It uses a string based domain specific language that you have to learn. After a while, you begin to feel a clumsiness when inserting variables into the constraints.
  • TableLayout – was a new approach to layouting by applying a global grid to your panel. You define the grid by specifying row and column constraints. If you need special cell constraints afterwards, you can alter them, but it’s getting bloated again.
  • StringTableLayout – provided a string based domain specific language over the TableLayout. It had some nice additional features, but lacked versatility with dynamic GUIs.
  • FormLayout – was a great relief and a good companion for many full sized layouting tasks. By concentrating on a problem domain (form based layouts), it played out some advantages over general purpose layout managers. This layout is still in use here.
  • MigLayout – the bigger brother of all these layouts. MigLayout comes with several pages of cheat sheets and you’re soon lost without it. It combines the approaches of all layout managers listed (and many more) and blends them into a massively powerful and versatile product. If you learn this layout manager thoroughly, you’ll never have to look elsewhere. But the learning curve is steep and the complexity of your code scales with the complexity of the GUI (which isn’t a drawback).

All these layout managers added value to our GUIs and are in use until today, albeit seldom.

Keep it simple

Most of the time, your dialogs aren’t these super-fancy, highly dynamic full-page layouts every UI designer dreams about. If they are, pick one of the layout managers from the list and wade through the constraint setup. But let’s say you want to layout a rather plain dialog with some widgets, but you want to do it quick without sacrificing the looks. Here is a developer-friendly solution for this task: Use the DesignGridLayout manager.

Slick and easy layouts

The one thing that differentiates the DesignGridLayout from almost every other layout manager is that you use the layout manager instance itself (in a fluent interface style) to arrange the constraints of your grid. You do not add your widgets to the panel and hope for the layout manager to catch up with the layout, you add them to the layout manager (and hope for it to fill it into your panel, which it does nicely). Here is a little example of the API usage:

JPanel content = new JPanel();
DesignGridLayout layout = new DesignGridLayout(content);
JTextArea history = new JTextArea();
history.setRows(5);
JTextField message = new JTextField();
JButton sendNow = new JButton("Send");
layout.row().grid(new JLabel("History:")).add(new JScrollPane(history));
layout.row().grid(new JLabel("Message:")).add(message, 2).add(sendNow);
content.setLayout(layout);

If you are interested in the possibilities of the layout manager, you should read the usage introduction page of DesignGridLayout.

Developer-friendly approach

One big advantage of the fluent API when compared with the string based constraint definitions is the compiler and type system support. You can’t spell anything wrong and the code completion feature of your IDE guides you to the right method and parameter order. The other advantage is that you don’t need to mess with pixel sizes for spacing and such. It’s handled by the layout manager in the most comfortable manner.

And because an article about a layout manager isn’t of any worth without a picture, here’s one:

This is a frame with the panel we constructed in the example code above.

Follow-up to our Dev Brunch August 2010

A follow-up to our August 2010 Dev Brunch, summarizing the talks and providing bonus material.

Last Sunday , we held our Dev Brunch for August 2010. We had to meet early in August, as there will be a lot of holiday absence in the next weeks. The setting was more classical again, with a real brunch on a late sunday morning. We had a lot more registrations than finally attendees, but it was said this was caused by a proper birthday party the night before. Due to rainy weather, we stayed inside and discussed the topics listed below.

The Dev Brunch

If you want to know more about the meaning of the term “Dev Brunch” or how we implement it, have a look at the follow-up posting of the brunch in October 2009. We continue to allow presence over topics. Our topics for the brunch were:

  • Clean Code Developer Initiative – The Clean Code Developer movement uses colored wristbands to subsequentially focus on different aspects of principles and practices of a professional software developer. Despite the name, it’s a german group with german web sites. But everybody who read Uncle Bob’s “Clean Code” knows what the curriculum is about. The talk gave a general summary about the intiative and some firsthand experiences with following the rules. If you read the book or are interested in profound software development, give it a try.
  • Non-bare repositories in git – The distributed version control system git differentiates between “bare” and “non-bare” repositories. If you are a local developer, you’ll use the non-bare type. When two developers with similar non-bare repositories (e.g. of the same project) meet, they can’t easily share commits or patches with the “push” command. This is a consequence of the “push” not being the exact opposite of the “fetch” command. If you try to synchronize two non-bare git repositories with push commands, you’ll most likely fail. The only safe approach is to introduce an intermediate bare repository or a branch in on of the repositories that only gets used by extern users. Even the repository owner has to push to this branch then. We discussed the setup and consequences, which are small in a broader use case and sad for ad-hoc workgroups.

Retrospection of the brunch

The group of attendees was small and a bit hung over. This led to a brunch that lacked technical topics a bit but emphasized social and cultural topics that didn’t make it on the list above. A great brunch just before the holiday season.