Does the following code run without an exception and what does the following code print out?
def isNull = null isNull += null println isNull.getClass()
Groovy Puzzler: null + null is?
Groovy handles null in a different way…
Code Camp Experiences
Experiences gained when performing a two day code camp with a team.
Some weeks ago, I conducted a code camp with a team of twelve developers that build a software product together for years now. The team had already introduced sporadic code reviews (in the team vs. author review style), so the main emphasize of the meeting was to improve team coherence by writing code together while generally having some time off project. In this article, I describe what I had planned, what happened and what the effects are so far.
A plan for the code camp
The code camp was scheduled for two consecutive days when the whole team gathers in one room with one computer for each pair. We would switch pairs (and seats) after each iteration, with one iteration being 45 minutes coding time followed by 2-3 minutes presentation of the achievement to the team. With a recreation break of 20-30 minutes, this means one iteration every two hours.
Every iteration starts from scratch, without access to previous code fragments (see also the code retreat concept). This had several reasons: I wanted the iterations to be comparable. Some of the insights I wanted to share are dependent on directly assimilable experiences. The iterations should also be independent, without ballast from previous sessions.
On the first day, we started with a given code resembling a little puzzle game in Java Swing. The code worked, but had some bugs and was written in an awful manner. It was unknown code for the team with no emotional attachments. The assignment for each iteration was to refactor the code to something equivalently working, but much “better”. How this “better” is defined was up to the teams.
On the second day, we started with a blank editor and had the task to code the same little puzzle game (in Java Swing) we refactored the day before. Even with some practice, it was nearly impossible to finish within time, so concentration on the most important key aspects of the code was crucial. The main lesson here was to “create from scratch” rather than “fix the existing”.
What really happened
Day one
The camp started with the usual delay for setting up all the computers in a uniform manner. This couldn’t be prepared beforehands, as the computers were in use by another group. When we installed our software, we found the hotkey configuration of the whole system severely flawed (for example, Ctrl+1 was defined as “set keyboard layout to traditional chinese”).
To warm up for the first iteration, I presented the existing code and explained its structure in detail. The code was slightly too much to remember it all in one pass, so only a rough understanding remained. Every team had to examine the code again during their work.
After the warm up, the first iteration started, with everybody buzzing over the code. The 45 minutes went by really fast and the first presentations focussed on local improvements. No team had restructured the code in any meaningful way, but every solution was perceived as “better” than the original. One team failed to get their refactored code to work again.
The second iteration held the biggest surprise of the whole camp. The 45 minutes flew by and the presentations showed the difference. One team failed to work on the assignment, but every other team presented a solution that was far superior of the original code. Some teams restructured the code to an extend where the original structure wasn’t recognizable anymore. The distinction between the first and the second iteration was so great that everybody was baffled by what could be achieved in 45 minutes when you do it for the second time.
The third iteration added some interesting twists on the best solutions of the second iteration, but didn’t produce the massive boost in productivity and code quality. Everybody felt worn out afterwards, so we decided to close the coding part of this day.
While working with the Java Swing code, nobody on the team noticed the threading flaws in the code. When I pointed this out, I was asked to explain the mechanics of the Swing threading model. The team develops a web application and hasn’t much exposure to desktop application development, let alone with Java Swing. So we ended the day with a lecture about the EDT, the EventQueue and the SwingWorker.
The whole team strolled to a bar to share some beers afterwards.
Day two
After a short night, we gathered early in the morning to continue the coding part of the camp. I explained the task (develop the game from scratch) and off we went.
The first iteration yielded only very rudimentary results. One team started with an UML diagram of the application structure and had to stop after setting up the outline of every method in code. Most other teams started with the domain model and failed to attach the GUI part of the application. All solutions had similar concepts in mind, no team used test driven development or other “advanced” techniques.
As a result of the poor performance, we decided to change the rules. Instead of scrapping the whole code, every new team could take over the code base of any other team, as long as it wasn’t the own. In the second iteration, we completed the drafted solutions of the previous team. This didn’t work out, too. The teams were frustrated by their lack of results.
We changed the plan again and held a prolonged review discussion of the code camp instead of a third iteration. This was by far the better choice with hindsight.
The effects
The code camp was perceived very positively by the attendees. The main goal of the camp was not about coding, but about team coherence and team focus. We learnt a lot about the personal style and abilities of each team member because the code samples shown in the code review were directly comparable. And we revealed team problems we weren’t aware of yet, but some problems we thought would arise did not. This was a very healthy process, because some of these issues can be addressed directly now.
The side benefit of the camp, as stated by one programmer was the increased awareness that “throwing away your code and starting over isn’t as hurtful as I thought”.
Every attendee stated that they want another code camp soon.
Personal summary
The code camp greatly improved my sense for the team and for the individual team members. By sharing a common code base and performing the same tasks, I could directly see (and feel) their thoughts and abilities. The camp is a powerful way to get to really know a team. If you have to mentor a whole new team, consider performing a code camp to get in touch with them.
Developing Grails Apps – Some Dark Sides
Most of the time, developing Grails apps is a nice experience. But there are also dark sides. One of which is when bugs do appear or do not appear depending on how you started your app.
Usually, I try to avoid it but this time a Disclaimer is in order: This is not a Grails rant. Most of the time developing Grails projects is fast and smooth. Using Grails brought many advantages for us. But there are also dark sides…
My main criticism is that Grails abstractions are more than leaky! In every list of examples for the definition of the term Leaky Abstraction Grails should be top. As soon as you leave the tutorial/scaffolding/helloworld level you have to know a lot about the underlying stack. And with Hibernate and Spring neither of the words small, easy and lightweight do apply.
GORM, too, is only easy to use at first sight. The very informative blog series about GORM gotchas should absolutely become part of the user guide or the refence docs.
And there are those times where it gets really unpleasant. This is e.g. when a bug does appear in your grails application running in a servlet container (packaged in a .war) but does not appear when the application is started from within the IDE. Our last one of those was a naming conflict in a .gsp file. The controller handed a model like this to the .gsp:
... return [fieldValue: 'THE_VALUE', ...]
The model entry ‘fieldValue’ was used in the .gsp to set the value of a combo box. Unfortunately, ‘fieldValue’ is also the name of a built-in Grails tag …
Admittedly, ‘fieldValue’ was not the wisest choice of names and I would certainly expect to get scolded loudly by Grails for that – ideally with a nice descriptive exception. But what happend instead led to a loud scolding of Grails from us. And to some big question marks: What is the difference between executing Grails from the IDE and within a servlet container with respect to naming resolution? Why is there a difference, at all?
We had a hard time figuring out this one, not least because the error message was not very telling. And since this was not the first of those works-in-the-IDE-but-not-in-a-real-environment bugs there is always this slightly uneasy feeling…
As I said in the beginning, most of the time developing Grails applications is nice and shiny. I would not support their slogan, though. My personal search for the best web development tool is definitively not over.
How about your search?
Combine cobertura with the awesomeness of crap4j
Want the awesomeness of crap4j without running your tests twice in your build? Just combine it with your cobertura data using crapertura.
You may have heard of crap4j when it was still actively developed. Crap4j is a software metric that points you to “crappy” methods in your projects by combining cyclomatic complexity numbers with test coverage data. The rationale is that overly complex code can only be tamed by rigorous testing or it will quickly reduce to an unmaintainable mess – the feared “rotten code” or “crappy code”, as Alberto Savoia and Bob Evans, the creators of crap4j would put it. The crap4j metric soon became our most important number for every project. It’s highly significant, yet easy to grasp and mandates a healthy coding style.
Some enhancements to crap4j
Crap4j got even better when we developed our own custom enhancements to it, like the CrapMap or the crap4j hudson plugin. We have a tool that formats the crap4j data like cobertura’s report, too.
A minor imperfection
The only thing that always bugged me when using crap4j inside our continuous integration build cycle was that at least half the data was already gathered. Cobertura calculates the code coverage of our tests right before crap4j does the same again. Wouldn’t it be great if the result of the first analysis could be re-used for the crap metric to save effort and time?
Different types of coverage
Soon, I learnt that crap4j uses the “path coverage” to combine it with the complexity of a method. This is perfectly reasonable given that the complexity determines the number of different pathes through the method. Cobertura only determines the “line coverage” and “branch coverage”. As it stands, you can’t use the cobertura data for crap4j because they represent different approaches to measure coverage. That’s still true and probably will be for a long time. But the allurement of the shortcut approach was too high for me to resist. I just tried it out one day to see the real difference.
A different metric
So, here it is, our new metric, heavily inspired by crap4j. I just took the line and branch coverage for every method and multiplied them. If you happen to have a perfect coverage (1.0 on both numbers), it stays perfect. If you only have 75% coverage on both numbers, it will result in a “crapertura coverage” of 56,25%. Then I fed this new coverage data into crap4j and compared the result with the original data. Well, it works on my project.
Presenting crapertura
Encouraged by this result, I wrote a complete ant task that acts similar to the original crap4j ant task. You can nearly use it as a drop-in replacement, given that the cobertura XML report file is already present. Here is an example ant call:
<crapertura coberturaReportFile="/path/to/cobertura/coverage.xml" targetDirectory="/where/to/place/the/crap4j/report" classesDirectory="/your/unarchived/project/class/files" />
It will output the usual crap4j report files to the given target directory. Please note that even if it looks like crap4j data, it’s a different metric and should be treated as such. Therefore, online comparison of numbers is disabled.
The whole project is published on github. Feel free to browse the code and compile it for yourself. If you want a binary release, you might grab the latest jar from our download server.
The complete usage guide can be found on the github page or inside the project. If you have questions or issues, please use the comment section here.
Conclusion
If crapertura is able to give you nearly the numbers that crap4j gave you is up to your project, really. Our test project contained over 20k methods, but very little crap. The difference between crap4j and crapertura was negligible. Both metrics basically identified the same methods as being crappy. Your mileage may vary, though. If that’s the case, let us know. If your experience is like ours, you’ve just saved some time in your build cycle without sacrificing quality.
Open Source Love Day September 2010
Our Open Source Love Day for September 2010 brought love for the cmake hudson plugin and a brand new tool around crap4j that will be released soon.
On wednesday last week, we held our Open Source Love Day for September 2010. Our day started with the usual Homepage Comittee meeting and very soon, we were up and working. This time, our success rate wasn’t as high as we wanted, mostly because we worked on internal tools that didn’t work out quite as well as expected. But, we managed to produce something valuable this day.
The Open Source Love Day
We introduced a monthly Open Source Love Day (OSLD) to show our appreciation to the Open Source software ecosystem and to donate back. We heavily rely on Open Source software for our projects. We would be honored if you find our contributions useful. Check out our first OSLD blog posting for details on the event itself.
On this OSLD, we accomplished the following tasks:
- A new version 1.6.1 of the cmakebuilder hudson plugin was published. This version consisted of bugfixes only and right now, it still seems flawed. We are working on the issue, expect a new version 1.6.2 soon.
- Our internal time-tracking tool got love at several issues. One issue required the use of triangle-shaping CSS, as described in this blog post from Jon Rohan. Our issue weren’t finished because Javascript code can rot into a big pile of crap really quick.
- We managed to make a long hatched dream come true at this OSLD. As you might be aware, we are big fanboys of crap4j, a metric tool that associates test coverage with code complexity. Thus, we wrote the crap4j hudson plugin, release the CrapMap and use some internal improvements, too. The main disadvantage of crap4j is the strong dependency on a specific test coverage tool. Our goal was to use the test coverage data we already collect using Cobertura. We achieved this goal and got the whole thing working. It will be released in the next few weeks, with a detailed blog post here. Stay tuned for this new tool (it already has a name: Crapertura).
What were our lessons learnt today?
- No matter how clever you are, Javascript outsmarts you every time it appears in superior numbers. Refactoring is the key here, but difficult and tedious to apply.
- When you dissect a foreign API or code base, you just need to find the right grip. I cannot decribe it more precise right now, but this grip is all you need to open up the code. When playing around with the crap4j code base, as soon as we held the grip, everything else followed naturally. Perhaps “the grip” can be translated with “catching the author’s intent”. These are always magical moments.
Retrospective of the OSLD
This OSLD wasn’t as successful as we wished, partly because of missing manpower (honeymoon holiday!) and because of our inability to tame a Javascript code base. We have to work on our expertise here and we are glad that we’ve found out at an OSLD, not a time- and mission-critical customer project.
GORM Gotchas: Validation and hasMany
Using validation on the end of hasMany associations yields unexpected results.
The excellent GORM Gotchas Series inspired me to write about a gotcha I found recently.
You have a domain class Container which contains elements:
class Container {
static hasMany = [elements:Element]
List<Element> elements
}
and the element has a constraint:
class Element {
static belongsTo = [container:Container]
static constraints = {
email(email: true)
}
String email
}
When you try to save a container with more than one element that fails validation, only the first error appears:
Container c = new Container() c.addToElements(new Element(email: "a")) c.addToElements(new Element(email: "b")) c.save() assertEquals(2, c.errors.allErrors.size()) // fails, only one error is recorded!
The solution described in the docs coined with In some situations (unusual situations)) is to use a custom validator in the container class:
class Container {
static constraints = {
elements(validator: { List val, Container obj ->
val.each {Element element ->
if(! element.validate()) {
element.errors.allErrors.each { error->
obj.errors.rejectValue(
'elements',
error.getCode(),
error.getArguments(),
error.getDefaultMessage()
)
}
}
}
return true
})
}
static hasMany = [elements:Element]
List<Element> elements
}
Why I give lectures in software engineering
I’m often asked why I give lectures in software engineering, as they appear to not pay off for me. I think they do and here is why.
For more than eight years, I give lectures in software engineering, object oriented programming and software development “best practices”. I have to spend nearly a day every week for six months in the year to prepare and hold the classes. My normal work schedule is always very stuffed with tasks, I have to affront my other duties sometimes in order to show up in front of the students. On many occassions, I’ve been asked why I keep giving lectures despite pressing liabilities, inferior payment and generally better alternatives elsewhere.
Here is a list with answers I’ve given to this questions over and over again. I do not want to convince you that giving lectures is the best thing right after sliced bread or that you will experience any of these if you manage to get in the same position. It’s just a rational explanation why the question still strikes me as odd.
- It’s pure fun – This surely doesn’t count for everyone, but for me, speaking (ranting, raving, arguing) about software development counts as fun times. Being “on stage” in front of the students helps me to free my thoughts from dead freight and completely concentrate on the topics.
- I’m being paid to recapitulate the basics – This are two advantages in one: being paid cannot be bad (albeit payment can always improve) and to repeat the basics of my craft on a regular schedule can be seen in the tradition of katas. I’m very bulletproof in discussions about fundamental topics of software engineering because I’ve heard most questions and had to answer them multiple times already.
- I’m constantly learning new facets about well-known topics – My students always bring in unique and original thoughts about topics that I thought to have mastered. And then, a new way to access things emerges, at least for me. I feel very certain that I’m still learning more during the lectures than my students do. And feedback suggests that they learn a lot.
- I’m honing my verbal abilities – Giving a lecture is all about speaking without script and responding to the audience. You have to make your points, but you cannot force them. Sometimes I feel like a stand-up comedian for technical knowledge. Having the ability to speak fluently while preparing the next topic in the back of your head is a great advantage in every situation including verbal communication.
- Roughly 100 aspiring developers remember me every year – What they will remember me for can be debated about, but they will remember. This is all about “networking”, but focussed on members of my own profession. The reach of this network amazes me every time when it loops back.
- I can contact every local company with job training – Due to the nature of the Cooperative State University where I’m giving my lectures, I can also establish contact to every software company in the vicinity. Many contacts would never happen without my function as lecturer.
- I keep in touch with hypes – Students are easy prey for IT hypes. Their experience with different technologies isn’t embittered by analogies from the past. All I have to do is to listen to them when they tell me about their work and hobby projects. And then I can draw my own conclusions based on their first-hand experience.
All these reasons and some more are enough for me to stick with the job. You can see a lot of short term benefits and some aspects that might pay off at medium term. On the long run, I’m convinced that my personal advantages from this job will outweight the (sometimes serious) drawbacks. And then, I haven’t yet included the advantages that my students took along from my lectures, hopefully.
If you happen to give lectures too, I would be pleased if you blog about your reasons for doing so, and announce your post here. Or just use the comment section.
Responsive Qt GUIs – Threading with Qt
Qt4 used to have only primitive threading support. Starting with version 4.4 new classes and functions makes your threading life a lot easier. So in case you haven’t come around to look at those features, do it now!, it’s worth it.
If you have used Qt4 for some time now, specifically since pre 4.4 versions, you may or may not aware of the latest developments in the threading part of the library. This post shall be a reminder in case you didn’t follow the versions in detail or just didn’t get around to look closer and/or update.
In pre 4.4 versions, the only way to do threading was to use class QThread. Subclass QThread, implement the run method, and there you had your thread. This looks fine at first, but, taking the title of the post as example, it can get annoying very fast. Sometimes you have just few lines of code you want to keep away from the GUI thread because, e.g. they could potentially block on some communication socket. Subclassing QThread for every small little work package is not something you want to do, so I guess many users just wrote their own thread pool or the like.
Starting with version4.4. Qt gained two major threading features, for which, IMHO, the Qt people do not a very good job of advertising. The first is QThreadPool together with QRunnable. All Java programmers, which use java.lang.Runnable since the beginning, may have their laugh now, I’ll wait…
The second new threading feature is the QtConcurrent namespace (from the Qt documentation):
The QtConcurrent namespace provides high-level APIs that make it possible to write multi-threaded programs without using low-level threading primitives such as mutexes, read-write locks, wait conditions, or semaphores
Sounds great! What else?
QtConcurrent includes functional programming style APIs for parallel list prosessing, including a MapReduce and FilterReduce implementation for shared-memory (non-distributed) systems, and classes for managing asynchronous computations in GUI applications.
This is really great stuff. Functions like QtConcurrent::run together with QFuture<T> and QFutureWatcher<T> can simplify your threading code significantly. So, if you haven’t got around to look at those new classes by now, I can only advise you to do it immediately. Allocate a refactoring slot in your next Sprint to replace all those old QThread sub-classes by shiny new QRunnables or QtConcurrent functions. It’s worth it!
Let’s get back to the responsive GUIs example. In his Qt Quarterly article, Witold Wysota describes in detail every technical possibility to keep your GUI responsive. It’s a very good article which provides a lot of insights. He starts with manual event processing and mentions the QtConcurrent features only at the very end of the article. I suggest the following order of threading-solutions-to-consider:
- QtConcurrent functions
- QThreadPool + QRunnable
- rest
Stay responsive!
Scala: Easier to read (and write), harder to understand?
There is a vivid discussion about Scala’s complexity going on for some weeks now on the web even with a response from Martin Odersky. I want to throw my 2¢ together with some hopefully new aspects into the discussion.
Scala’s liberal syntax rules and compiler magic like type inference and implicit conversions allow nicely written APIs and DSLs almost looking like prose texts. Take a look at APIs like scalatest and imagine the Java/Junit equivalent:
@Test def demonstrateScalaTest() {
val sb = new StringBuilder("Scala")
sb.append(" is cool!")
sb.toString should be ("Scala is cool!")
evaluating { "concise".charAt(-1) } should produce [StringIndexOutOfBoundsException]
}
There are really nice features that reduce day-to-day programming tasks to keywords or one-liners. Here are some examples:
// singletons have their own keyword (object), static does not exist!
object MySingleton {
def printMessage {
println("I am the only one")
}
}
// lazy initialization/evaluation
lazy val complexResult = computeForHours()
// bean-style data container with two scala properties and one java-bean property with getter+setter
class Data(val readOnly: String, var readWrite: Int, @BeanProperty var javaProperty: String)
// tuples as return values or quick data transfer objects (DTO) for methods yielding multiple data objects
def correctCoords(x: Double, y: Double) = (x + 12, y * 0.5)
val (correctedX, correctedY) = correctCoords(0.37, 34.2)
println("corrected: " + correctedX + ", " + correctedY)
On the other hand there are so many features built-in that really make it hard to understand the code if you are not scala programmer with some experience. I like the differentiation between application and library code Martin Odersky himself makes in Programming Scala. The frameworks I have tried so far (Lift, scalatest and scala-swing) in Scala make your life very easy as long as you just use them. It is really a breeze and much more fun than using most APIs in Java for example. But when something goes wrong or you really want/have to understand what is going on you can have a hard time. This is true at least for a Scala beginner, sometimes perhaps for an pro, too.
Final Thoughts
In my opinion Scala is a very nice language that successfully combines clean object oriented programming with functional features. You can migrate from a pure OO-style to a nice hybrid “Scala-style” like many programmers did when they first used Java mostly with procedural style using classes only as namespaces for their static methods. I am quite sure that a Scala code style and best practices still have to develop. Programmers will need their time diving into the language and using it for their benefit. I hope Scala prospers and gains attention in the industry because I personally think it is a nice step forward compared to Java (which turns more and more into a mess where you need profound knowledge to fight your problems).
Regarding the complexity, which certainly exists in Scala, I only want to raise some questions which may be answered sometime in the future:
- Maybe the tooling is just not there (yet)?
- Maybe you sometimes just don’t have to understand everything what’s happening underneath?
- Maybe Scala makes debugging much more seldom but harder, when something does not work out?
- Maybe the features and power of Scala are worth learning?
- Maybe certain features will just be banned by the teams like sometimes in Java teams (think of switch-case, the ?-operator, Autoboxing e.g.)?
== isn’t equals, or is it?
Beware of the subtle differences of == and equals in Java and Groovy.
== and equals behave different in Java (and Groovy). You all know the subtle difference when it comes to comparing strings. equals is recommended in Java, == works in Groovy, too. So you could think that using equals is always the better option… think again!
Take a look at the following Groovy code:
String test = "Test"
assertTrue("Test" == test) // true!
assertTrue("Test" == "${test}") // true!
assertTrue("Test".equals("${test}")) // false?!
The same happens with numbers:
assertTrue(1L == 1) // true! assertTrue(1L.equals(1)) // false?!
A look at the API description of equals shows the underlying cause (taken from the Integer class):
Compares this object to the specified object. The result is true if and only if the argument is not null and is an Integer object that contains the same int value as this object.
equals follows the contract or best practice (see Effective Java) that the compared objects must be of the same class. So comparing different types via equals always results in false. You can convert both arguments to the same type beforehand to ensure that you get the expected behavior: the comparison of values. So next time when you compare two values think of the types or convert both values to the same type.
Schneide Blog 