Responsive Qt GUIs – Threading with Qt

Qt4 used to have only primitive threading support. Starting with version 4.4 new classes and functions makes your threading life a lot easier. So in case you haven’t come around to look at those features, do it now!, it’s worth it.

If you have used Qt4 for some time now, specifically since pre 4.4 versions, you may or may not aware of the latest developments in the threading part of the library. This post shall be a reminder in case you didn’t follow the versions in detail or just didn’t get around to look closer and/or update.

In pre 4.4 versions, the only way to do threading was to use class QThread. Subclass QThread, implement the run method, and there you had your thread. This looks fine at first, but, taking the title of the post as example, it can get annoying very fast. Sometimes you have just few lines of code you want to keep away from the GUI thread because, e.g. they could potentially block on some communication socket. Subclassing QThread for every small little work package is not something you want to do, so I guess many users just wrote their own thread pool or the like.

Starting with version4.4. Qt gained two major threading features, for which, IMHO, the Qt people do not a very good job of advertising. The first is QThreadPool together with QRunnable. All Java programmers, which use java.lang.Runnable since the beginning, may have their laugh now, I’ll wait…

The second new threading feature is the QtConcurrent namespace (from the Qt documentation):

The QtConcurrent namespace provides high-level APIs that make it possible to write multi-threaded programs without using low-level threading primitives such as mutexes, read-write locks, wait conditions, or semaphores

Sounds great! What else?

QtConcurrent includes functional programming style APIs for parallel list prosessing, including a MapReduce and FilterReduce implementation for shared-memory (non-distributed) systems, and classes for managing asynchronous computations in GUI applications.

This is really great stuff. Functions like QtConcurrent::run together with QFuture<T> and QFutureWatcher<T> can simplify your threading code significantly. So, if you haven’t got around to look at those new classes by now, I can only advise you to do it immediately. Allocate a refactoring slot in your next Sprint to replace all those old QThread sub-classes by shiny new QRunnables or QtConcurrent functions. It’s worth it!

Let’s get back to the responsive GUIs example. In his Qt Quarterly article, Witold Wysota describes in detail every technical possibility to keep your GUI responsive. It’s a very good article which provides a lot of insights. He starts with manual event processing and mentions the QtConcurrent features only at the very end of the article. I suggest the following order of threading-solutions-to-consider:

  1. QtConcurrent functions
  2. QThreadPool + QRunnable
  3. rest

Stay responsive!

Scala: Easier to read (and write), harder to understand?

There is a vivid discussion about Scala’s complexity going on for some weeks now on the web even with a response from Martin Odersky. I want to throw my 2ยข together with some hopefully new aspects into the discussion.
Scala’s liberal syntax rules and compiler magic like type inference and implicit conversions allow nicely written APIs and DSLs almost looking like prose texts. Take a look at APIs like scalatest and imagine the Java/Junit equivalent:

@Test def demonstrateScalaTest() {
  val sb = new StringBuilder("Scala")
  sb.append(" is cool!")
  sb.toString should be ("Scala is cool!")
  evaluating { "concise".charAt(-1) } should produce [StringIndexOutOfBoundsException]
}

There are really nice features that reduce day-to-day programming tasks to keywords or one-liners. Here are some examples:

// singletons have their own keyword (object), static does not exist!
object MySingleton {
  def printMessage {
    println("I am the only one")
 }
}

// lazy initialization/evaluation
lazy val complexResult = computeForHours()

// bean-style data container with two scala properties and one java-bean property with getter+setter
class Data(val readOnly: String, var readWrite: Int, @BeanProperty var javaProperty: String)

// tuples as return values or quick data transfer objects (DTO) for methods yielding multiple data objects
def correctCoords(x: Double, y: Double) = (x + 12, y * 0.5)
val (correctedX, correctedY) = correctCoords(0.37, 34.2)
println("corrected: " + correctedX + ", " + correctedY)

On the other hand there are so many features built-in that really make it hard to understand the code if you are not scala programmer with some experience. I like the differentiation between application and library code Martin Odersky himself makes in Programming Scala. The frameworks I have tried so far (Lift, scalatest and scala-swing) in Scala make your life very easy as long as you just use them. It is really a breeze and much more fun than using most APIs in Java for example. But when something goes wrong or you really want/have to understand what is going on you can have a hard time. This is true at least for a Scala beginner, sometimes perhaps for an pro, too.

Final Thoughts
In my opinion Scala is a very nice language that successfully combines clean object oriented programming with functional features. You can migrate from a pure OO-style to a nice hybrid “Scala-style” like many programmers did when they first used Java mostly with procedural style using classes only as namespaces for their static methods. I am quite sure that a Scala code style and best practices still have to develop. Programmers will need their time diving into the language and using it for their benefit. I hope Scala prospers and gains attention in the industry because I personally think it is a nice step forward compared to Java (which turns more and more into a mess where you need profound knowledge to fight your problems).

Regarding the complexity, which certainly exists in Scala, I only want to raise some questions which may be answered sometime in the future:

  • Maybe the tooling is just not there (yet)?
  • Maybe you sometimes just don’t have to understand everything what’s happening underneath?
  • Maybe Scala makes debugging much more seldom but harder, when something does not work out?
  • Maybe the features and power of Scala are worth learning?
  • Maybe certain features will just be banned by the teams like sometimes in Java teams (think of switch-case, the ?-operator, Autoboxing e.g.)?

== isn’t equals, or is it?

Beware of the subtle differences of == and equals in Java and Groovy.

== and equals behave different in Java (and Groovy). You all know the subtle difference when it comes to comparing strings. equals is recommended in Java, == works in Groovy, too. So you could think that using equals is always the better option… think again!
Take a look at the following Groovy code:

  String test = "Test"
  assertTrue("Test" == test) // true!
  assertTrue("Test" == "${test}") // true!
  assertTrue("Test".equals("${test}")) // false?!

The same happens with numbers:

  assertTrue(1L == 1) // true!
  assertTrue(1L.equals(1)) // false?!

A look at the API description of equals shows the underlying cause (taken from the Integer class):

Compares this object to the specified object. The result is true if and only if the argument is not null and is an Integer object that contains the same int value as this object.

equals follows the contract or best practice (see Effective Java) that the compared objects must be of the same class. So comparing different types via equals always results in false. You can convert both arguments to the same type beforehand to ensure that you get the expected behavior: the comparison of values. So next time when you compare two values think of the types or convert both values to the same type.

Java Swing Layouting done right

A praise of the most developer-friendly Java Swing layout manager to date: DesignGridLayout.

Layout Managers were an huge benefit for Java Swing. They enabled software developers to program layout rather than to “drag and drop” it with some proprietary GUI builder. That’s nothing against a good GUI builder, but against the “source code” that gets generated as a result of using it. But after some time of playing and working with the layout managers given by Swing itself, we concluded that they weren’t up to the task. Since then, we were constantly on the lookout for new and better ways to tackle the layouting task.

A history of layout managers

Let’s reiterate our major path with different layout managers:

  • GridBagLayout – the most versatile layout manager included in the Java Swing core classes. It’s capable to handle virtually every layouting task, but the price is huge constraint setup code. Since the code bloats with even facile complexity in the dialog, it’s not maintainable once written. The advantages over GUI builders aren’t really present.
  • StringGridBagLayout – has the same power as GridBagLayout, but with much more concise constraint definitions. It uses a string based domain specific language that you have to learn. After a while, you begin to feel a clumsiness when inserting variables into the constraints.
  • TableLayout – was a new approach to layouting by applying a global grid to your panel. You define the grid by specifying row and column constraints. If you need special cell constraints afterwards, you can alter them, but it’s getting bloated again.
  • StringTableLayout – provided a string based domain specific language over the TableLayout. It had some nice additional features, but lacked versatility with dynamic GUIs.
  • FormLayout – was a great relief and a good companion for many full sized layouting tasks. By concentrating on a problem domain (form based layouts), it played out some advantages over general purpose layout managers. This layout is still in use here.
  • MigLayout – the bigger brother of all these layouts. MigLayout comes with several pages of cheat sheets and you’re soon lost without it. It combines the approaches of all layout managers listed (and many more) and blends them into a massively powerful and versatile product. If you learn this layout manager thoroughly, you’ll never have to look elsewhere. But the learning curve is steep and the complexity of your code scales with the complexity of the GUI (which isn’t a drawback).

All these layout managers added value to our GUIs and are in use until today, albeit seldom.

Keep it simple

Most of the time, your dialogs aren’t these super-fancy, highly dynamic full-page layouts every UI designer dreams about. If they are, pick one of the layout managers from the list and wade through the constraint setup. But let’s say you want to layout a rather plain dialog with some widgets, but you want to do it quick without sacrificing the looks. Here is a developer-friendly solution for this task: Use the DesignGridLayout manager.

Slick and easy layouts

The one thing that differentiates the DesignGridLayout from almost every other layout manager is that you use the layout manager instance itself (in a fluent interface style) to arrange the constraints of your grid. You do not add your widgets to the panel and hope for the layout manager to catch up with the layout, you add them to the layout manager (and hope for it to fill it into your panel, which it does nicely). Here is a little example of the API usage:

JPanel content = new JPanel();
DesignGridLayout layout = new DesignGridLayout(content);
JTextArea history = new JTextArea();
history.setRows(5);
JTextField message = new JTextField();
JButton sendNow = new JButton("Send");
layout.row().grid(new JLabel("History:")).add(new JScrollPane(history));
layout.row().grid(new JLabel("Message:")).add(message, 2).add(sendNow);
content.setLayout(layout);

If you are interested in the possibilities of the layout manager, you should read the usage introduction page of DesignGridLayout.

Developer-friendly approach

One big advantage of the fluent API when compared with the string based constraint definitions is the compiler and type system support. You can’t spell anything wrong and the code completion feature of your IDE guides you to the right method and parameter order. The other advantage is that you don’t need to mess with pixel sizes for spacing and such. It’s handled by the layout manager in the most comfortable manner.

And because an article about a layout manager isn’t of any worth without a picture, here’s one:

This is a frame with the panel we constructed in the example code above.

Non-trivial Custom Data in QActions

If you want to implement dynamic context menus with non-trivial custom data in your QActions, the Qt4 documentation is not very helpful. The article describes some solutions to this task.

Sometimes I get very frustrated with the online Qt4 documentation. Sure, the API docs are massive but for many parts they provide only very basic information. Unfortunately, many Qt books, too, often stop exactly at the point where it gets interesting.

One example for this are context menus. The API docs just show you how menus in general are created and how they are connected to the application: Basically, all menus are instances of QMenu which are filled with instances of QAction. QActions are used as representation of any kind of action than can be triggered from the GUI.

The standard method to connect QActions to the GUI controlling code is to use one of their signals, e.g. triggered(). This signal can be connected to a slot of your own class where you can then execute the corresponding action. This works fine as long as you have a limited set of actions that you all know at coding time. For example, a menu in your tool bar which contains actions Undo/Redo/Cut/Copy/Paste can be created very easily.

But there are use cases where you do not know in advance how many actions there will be in your menus. For example, in an application that provides a GUI for composing a complex data structure you may want to provide the user assisting context menus for adding new data parts depending on what parts already exist. Suddenly, you have to connect many actions to one slot and then you somehow have to know which QAction the user actually clicked.

Btw, let’s all recall the Command Pattern for a moment… ok, now on to some solutions.

Method 0 – QAction::setData: The QAction class provides method setData(), which can be used to store custom data in a QAction instance using QVariant as data wrapper. If you then use QMenu’s triggered signal, which gives you a pointer to the QAction that was clicked, you can extract your data from the QAction. I find this a little bit ugly since you have to wrap your data into QVariant which can get messy, if you want to provide more than one data element

Method 1 – Enhancing QAction::triggered(): By sub-classing QAction you can provide your own triggered() signal which you can enhance with all parameters you need in your slot.

class MyAction : public QAction
{
  Q_OBJECT
  public:
    MyAction(QString someActionInfo)
      : someActionInfo_(someActionInfo)
    {
      connect(this, SIGNAL(triggered()),
              this, SLOT(onTriggered()));
    }
  signals:
    void triggered(QString someActionInfo);
  private slots:
    void onTriggered() {
      emit triggered(someActionInfo_);
    }
  private:
    QString someActionInfo_;
};

This is nice and easy but limited to what data types can be transported via signal/slot parameters.

Method 2 – QSignalMapper: From the Qt4 docs on QSignalMapper:

This class collects a set of parameterless signals, and re-emits them with integer, string or widget parameters corresponding to the object that sent the signal.

… which is basically the same as we did in method 1.

Method 3 – Separate domain specific action classes: By the time the context menu is created you add QActions to the menu using QMenu’s addAction methods. Then you create instances of separate Command-like classes (as in Command Pattern) and connect them with the QAction’s triggered() signal:

// Command-like custom action class. No GUI related stuff here!
class MySpecialAction : public QObject
{
  Q_OBJECT
  public:
    MySpecialAction(QObject* parent, &lt;all necessary parameters to execute&gt;);

  public slots:
    void execute();
  ...
};

// create context menu
QAction* specialAction =
  menu-&gt;addAction(&quot;Special Action Nr. 1&quot;);
MySpecialAction* mySpecialAction =
  new MySpecialAction(specialAction, ...);
connect(specialAction, SIGNAL(triggered()),
        mySpecialAction, SLOT(execute()));

As you can see, QAction specialAction is parent of mySpecialAction, thereby taking ownership of mySpecialAction. This is my preferred approach because it is the most flexible in terms of what custom data can be stored in the command. Furthermore, the part that contains the execution code – MySpecialAction – has nothing at all to do with GUI stuff and can easily be used in other parts of the system, e.g. non-GUI system interfaces.

How have you solved this problem?

Grails pitfall: Proxies and inheritance

When using inheritance in a lazy fetched association, proxies are a common pitfall in Grails.

Problem

You have a domain class hierarchy and want to use the classes in an association. You typically use the base class in a hasMany:

class Domain {
  static hasMany = [others:BaseClass]
}

Dependent on the actual class in the set others you want to call a different method:

public doSomething(domain) {
  domain.others.each {doThis(it)}
}

public doThis(ChildClassOne c) {}

public doThis(ChildClassTwo c) {}
...

Here the proxy mechanism of Hibernate (and Grails) causes a MethodMissingException to be thrown because the proxies are instances of the base class and not the actual child ones.
One way would be to activate eager fetching or you could use…

Solution: A dynamic visitor

Declare a visit method in the base class which takes a closure

def visit(Closure c) {
return c(this)
}

and make the dispatch in a method for the base class:

public doThis(BaseClass c) {
  return c.visit {doThis(it)}
}

Add flair to your code: Code Squiggles

Introduction to Code Squiggles, a coding style that improves the readability of your java code.

Wrought Iron by quadriremeFor several months now, I’m experimenting with a programming style that you might want to call “syntax aware programming”. Every coding step starts with the question “what do I want to achieve and how do I want to write it down?”. Then I proceed to write it down in this perfect manner, mostly some english sentence, and try to incrementally convert the syntax into something the java compiler stops complaining about. There’s a lot to be learnt about API design, naming and creative syntax usage this way. One thing I’ve discovered along the way are Code Squiggles.

Introduction to Code Squiggles

Code squiggles are little methods that contain no functionality, other than directly returning the single given parameter. Their purpose is to make the code more fluently readable by casual readers. Calling these methods is absolutely optional, as they offer no behaviour at runtime. But by bloating your code with these method calls, you lower the amount of syntax rules and conventions the reader has to know before being able to understand the code. The process of adding the method calls feels like adding “happy little squiggles” (I certainly miss Bob Ross!) to your code.

Adding Code Squiggles by example

Let me give you a full example how Code Squiggles can turn your boring old java code into something even non-programming readers can grasp without problems. Note that the process of transforming the code isn’t the process I initially described, but the way to deal with existing code.

The initial code fragment looks like this:

assertTrue(filesDirectory.getChild("0.png").isFile());

As you can see, it’s an assertion line of an unit test. We improve the readability by extracting the intent of the assertion into the called method name (it’s a normal “extract method” refactoring):

assertFileExists("0.png", filesDirectory);

Now is the time to add the first Code Squiggle:

assertFileExists("0.png", in(filesDirectory));

You’ve noticed the difference? It’s just the word “in”, but it improves the semantics of the second parameter. As I promised, Code Squiggles are methods without any functionality worth talking about:

protected VirtualFile in(VirtualFile filesDirectory) {
    return filesDirectory;
}

The method takes a parameter and returns it. The real “functionality” of this method lies within its name. Everything else is only necessary to overcome (or overcode) java’s shortcoming of advanced syntax definition measures.

That’s all about Code Squiggles. But it doesn’t stop here. Let’s improve the example to its final shape, when we add two squiggles:

assertFile(withName("0.png"), existsIn(filesDirectory));

Read this line out loud! And notice the subtle change in the assertion method’s name. It begins to deminish clarity without the Code Squiggles. That’s when your API begins to depend on them. But it’s still perfectly valid java code if you just omit them.

The conceptual origin of Code Squiggles clearly comes from the behaviour driven development (BDD) style of writing tests and the ScalaTest Matchers. So I can’t claim much originality or cleverness myself here. But Code Squiggles, despite this initial example, aren’t limited to test code.

Adding inline documentation with Code Squiggles

Remember the last time you needed to integrate an horrible third-party API? Probably, there was a method with seven or eight parameters and all of them were primitive ints. No matter how often you called this method, you couldn’t remember the order of these ints. That’s when Code Squiggles might help you a bit.

This is the signature of a fairly complex method:

protected int performCalculation(int value, int lowerLimit, int upperLimit, int offset) { ...

Therefore, your call isn’t very self-explanatory:

int result = performCalculation(10, 2, 23, 13);

But it might be a lot more understandable when you add some squiggles:

int result = performCalculation(value(10), lowerLimit(2), upperLimit(23), offset(13));

I won’t spell out the squiggle implementations, as they should be straight-forward. Note that the java compiler doesn’t catch up here. You can easily swap the squiggles around to obfuscate your code. All you got is read-time safety. If you want compile-time safety, you need to replace the primitives with types and probably pay for some rounds of beer for the third-party API developers to include your changes or write an adapter (“corruption layer” is my preferred term here).

Regular use of Code Squiggles

If you want to use squiggles a lot, you might think about collecting them in a dedicated class. Design them to be static and generic, and you can use them easily with static imports:

public static <T> T existsIn(T instance) {
    return instance;
}

But remember that there are reserved keywords in java that might hamper you a bit.

Conclusion

Code Squiggles are useless bloat to your code unless you value read-time safety or casual readability. They can tidy up your code to a point where method or even class names are affected to form a block of code where you only have to replace the funnier chars with blanks to obtain a paragraph of plain written english anybody can read and understand.

What are your ideas towards Code Squiggles? Have you used them on your code? Mind to share the result?

Code squiggles are little methods that contain no functionality, other than directly returning the single given parameter. Their purpose is to make the code more fluently readable by casual readers.

The C++ Shoot-yourself-in-the-Foot of the Week

I think we can all agree that C++, compared to other languages, provides quite a number of possibilities to screw up. Everybody working with the language at some point probably had problems with e.g. its exception system, operator overloading or automatic type conversions – to name just a few of the darker corners.

There are also many mitigation strategies which all come down to ban certain aspects of the language and/or define strict code conventions. If you follow Google’s style guide, for example, you cannot use exceptions and you are restricted to a very small set of boost libs.

But developers – being humans – often find creative and surprising ways to thwart every good intentions. In an external project the following conventions are in place:

  • Use const wherever you possibly can
  • Use boost::shared_ptr wherever it makes sense.
  • Define typedefs to shared_ptrsย  in order to make code more readable.
  • typedefs to shared_ptrs are to be defined like this:
typedef boost::shared_ptr&lt;MySuperDuperClass&gt; MySuperDuperClassPtr;
  • typedefs to shared const pointers are to be defined like this:
typedef boost::shared_ptr&lt;const MySuperDuperClass&gt; MySuperDuperClassCPtr;

As you can see, postfixes Ptr and CPtr are the markers for normal shared_ptrs and constant shared_ptrs.

Last week, a compile error about some const to non-const conversion made me nearly pull my hair out. The types of variables that were involved all had the CPtr postfix but still the code didn’t compile. After a little digging I found that one of the typedefs involved was like this:

typedef boost::shared_ptr&lt;  MySuperDuperClass&gt; MySuperDuperClassCPtr;

Somebody just deleted the const modifier in front of MySuperDuperClass but left the name with the CPtr untouched. And because non-const to const conversions are allowed this was not detected for a couple of weeks. Nice going!

Any suggestions for a decent style checker for c++? Thanks in advance ๐Ÿ˜‰

Wrestling with Qt’s Model/View API – Filtering in Tree Models

Qt4’s model/view API can be kind of a challenge sometimes. Well, prepare for a even harder fight when sorting and filtering come into play.

As I described in one of my last posts, Qt4’s model/view API can be kind of a challenge sometimes. Well, prepare for a even harder fight when sorting and filtering come into play.

Let’s say you finally managed to get the data into your model and to provide correct implementations of the required methods in order for the attached views to display it properly. One of your next assignments after that is very likely something like implementing some kind of sorting and filtering of the model data. Qt provides a simple-at-first-sight proxy architecture for this with API class QSortFilterProxyModel as main ingredient.

Small preliminary side note: Last time I checked it was good OO practice to have only one responsibility for a given class. And wasn’t that even more important for good API design? Well, let’s not distract us with such minor details.

With my model implementation, none of the standard filtering mechanisms, like setting a filter regexp, were applicable, so I had to override method

QVariant filterAcceptsRow ( int source_row, const QModelIndex& sourceParent ) const

in order to make it work. Well, the rows disappeared as they should, but unfortunately so did all the columns except the first one. So what to do now? One small part of the documentation of QSortFilterProxyModel made me a little uneasy:

“… This simple proxy mechanism may need to be overridden for source models with more complex behavior; for example, if the source model provides a custom hasChildren() implementation you should also provide one in the proxy model.”

What on earth should I do with that? “… may need to be overridden“? “… for example.. hasChildren()…” Why can’t they just say clearly what methods must be overridden in which cases???

After a lot more trial and error I found that for whatever reason,

int columnCount ( const QModelIndex& parent ) const

had to be overridden in order for the columns to reappear. The implementation looks like what I had thought the proxy model would do already:

int MyFilter::columnCount ( const QModelIndex& parent ) const
{
   return sourceModel()->columnCount(parent);
}

So beware of QSortFilterProxyModel! It’s not as easy to use as it looks, and with that kind of fuzzy documentation it is even harder.

About PrintStream and Exceptions

Several of our projects deal with sensor hardware of different types often connected via the good oldโ„ข serial port. That is fine most of the time because most protocols are simple and RXTX provides a nice cross-platform library for most of your serial port needs. But many new computers do not feature the old RS232 serial ports anymore or other contraints prevent the use of a plain RS232 serial port. Here come serial converters like the Advantech ADAM 4570 (serial-to ethernet) or usb-to-serial converters into play. Usually this works fine.

Now one of our customers had a test system using an unreliable converter with sensor hardware. The hardware problems uncovered a robustness issue in our software which crashed the JVM when the virtual serial port of the converter disappeared and our app tried to write to it. Despite the faulty hardware our software had to be robust because it manages many more devices other than just that one sensor over serial. Looking at the problem we discovered that the crash occurred somewhere in the native part of RXTX. So we decided to scratch our own itch (and the one of the customer) and set out to fix the issue in RXTX at a Open Source Love Day (OSLD) . So we fixed the problem and submitted the patch to the bugtracker of the RXTX project. Our sample program now worked flawlessly and threw an IOException when the serial port failed in some way.

Happy to have fixed the problem we incorporated the patch RXTX in our production software but it still crashed and no IOException appeared anywhere in the logs. After another bughunting session we spotted the subtle difference of sample and production program: the use of OutputStream insted of PrintStream. PrintStream silently swallows all exceptions which proved fatal in our use case with the unreliable stream carrier. So the final fix was essentially replacing our PrintStream code

RXTXPort port = new RXTXPort("COM6");
PrintStream p = new PrintStream(port.getOutputStream(), true, "iso8859");
p.print("command");

with using OutputStream directly:

RXTXPort port = new RXTXPort("COM6");
OutputStream o = port.getOutputStream();
o.write("command".getBytes("iso8859"));

Conclusion

Be careful when using PrintStream with unreliable stream carriers it swallows exceptions! That may shadow problems which you may want to know of. Often PrintStreams behaviour will not be a problem but in certain cases like the one depicted above it causes a lot of headaches.