Category: Programming Language

Spice up your unit testing

Writing unit tests shouldn’t be a chore. This article presents six tools (with alternatives) that help to improve your developer experience.

Writing unit tests is an activity every reasonable developer does frequently. While it certainly is a useful thing to do, it shouldn’t be a chore. To help you with the process of creating, running and evaluating unit tests, there are numerous tools and add-ons for every programming language around. This article focusses on improving the developer experience (the counterpart of “user experience”) for Java, JUnit and the Eclipse IDE. I will introduce you to the toolset we are using, which might not be the complete range of tools available.

Creating unit tests

  • MoreUnit – This plugin for Eclipse helps you to organize your unit test classes by maintaining a connection between the test and the production class. This way you’ll always see which classes and methods still lack a corresponding test. You can take shortcuts in the navigation by jumping directly into the test class and back. And if you move one file, MoreUnit will move the other one alongside. It’s a swiss army knife for unit test writers and highly recommended.
  • EqualsVerifier – If you ever wrote a custom implementation of the equals()/hashcode() method pair, you’ll know that it’s not a triviality. What’s even more intimidating is that you probably got it wrong or at least not fully correct. The effects of a flawed equals() method aren’t easily determinable, so this is a uncomfortable situation. Luckily, there is a specialized tool to help you with this task exactly. The EqualsVerifier library tests your custom implementation against all aspects of the art of writing an equals() method with just one line of code.
  • Mockito (and EasyMock) – When dealing with dependencies of classes under test, mock objects can come in handy. But writing them by hand is tedious, boring and error-prone. This is where mock frameworks can help by reducing the setup and verification of a mock object to just a few lines of code. EasyMock is the older of the two projects, but it manages to stay up-to-date by introducing new features and syntax with every release. Mockito has a very elegant and readable syntax and provides a rich feature set. There are other mock frameworks available, too.

Running unit tests

  • InfiniTest (and JUnit Max) – Normally, you have to run the unit tests in your IDE by manually clicking the “run” button or hitting some obscure keyboard shortcut. These two continuous testing tools will run your tests while you still type. This will shorten your test feedback loop to nearly milliseconds after each change. Your safety net was never closer. InfiniTest and JUnit Max are both Eclipse plugins, but the latter costs a small annual fee. It’s written by Kent Beck himself, though.

Evaluating unit tests

  • EclEmma (and Cobertura) – If you want to know about the scope or “coverage” of your tests, you should consult a code coverage tool. Cobertura produces really nice HTML reports for all your statistical needs. EclEmma is an Eclipse plugin that integrates the code coverage tool Emma with Eclipse in the finest way possible. Simply run “coverage as” instead of “run as” and you are done. All the hassle with instrumenting your classes and setting up the classpath in the right order (major hurdles when using cobertura) is dealt with behind the scenes.
  • Jester (and Jumble) – The question “who tests my tests?” is totally legit. And it has an answer: Every mutation testing tool around. For Java and JUnit, there are at least two that do their job properly: Jester works on the source code while Jumble uses the bytecode. Mutation testing injects little changes into your production code to test if your tests catch them. This is a different approach on test coverage that can detect code that is executed but not pinned down by an assertion. While Jester has a great success story to tell, Jumble tends to produce similar results as cobertura’s condition coverage report, at least in my experience.

Summary

As you can see, there is a wide range of tools available to improve your efforts to write well-tested software. This list is in no way comprehensive. If you know about a tool that should be mentioned, we would love to read your comment.

Reversing an array in Java

Reversing an array is a popular interview question especially in languages like C. Some days ago I faced the problem in a Java legacy project I was maintaining. As a Java guy I did not want to fiddle with a for-loop and indexes. So I looked for another solution. Besides showing the one-liner I want to give some insights which some might not be aware of. Here is my solution:

public static <T> T[] reverse(T[] array) {
    Collections.reverse(Arrays.asList(array));
    return array;
}
  1. This approach works, because the ArrayList implementation Arrays.asList() returns a specially tailored ArrayList which writes the changes right through to the backing array. It is not a java.util.ArrayList!
  2. The above means that the array is changed in-place and no array copying is involved. The approach thus has good performance it that matters.
  3. My solution directly changes the given array. To make it free of side-effects you could create a new array and copy the original one into it before converting to a list and reversing.

Side note regarding Arrays.asList()

Arrays.asList() is nice for bridging to old code and APIs. It has to be used with caution though: If you read the API documentation and keep 1.) in mind you will notice, that the returned list is fixed size. So calls to add() and remove() will fail with an UnsupportedOperationException! Passing such lists around in your system using the java.util.List interface may lead to unexpected behaviour since most people expect lists to be of variable sized in Java. In our use case we do not return the resulting list to the caller but only use it temporarily and locally. So this is no problem here.

Conclusion

Reversing an array can be a nice interview question for Java candidates too as you can discuss about in-place reversing, memory and time complexity and different approaches. It may show their knowledge of the collection API and the utility classes. If the above approach is discovered there is also something to discuss as depicted in this article.

Grails: The good, the bad, the ugly

(Opinion!) After 3 years of Grails development it is time to take a step back and look how well we went.

After 3 years of Grails development it is time to take a step back and look how well we went.
(Info: we made several Grails apps ranging from small (<15 domain classes) to medium sized (50-70 domain classes) using front ends like Flex/Flash and AJAX)

The good parts

Always start with praise. So I tell you what in my opinion was and is good about developing in Groovy and Grails.

Groovy is Java with sugar

The Groovy syntax and the type system are so close to Java, so that when you come from a Java background you feel right at home.

Standard web stack

If you are accustomed to standard technologies like Spring and Hibernate you see Grails as a vacation.

Sensible defaults aka Convention over configuration

Many of the configuration options are filled with sensible defaults.

Fast start

You get from 0 to 100 in almost no time.

The bad things

Things which are not easily avoided.

Bugs, bugs, bugs

Grails has many, many bugs, unfortunately even in such fundamental things such as data binding and validation. A comment from a previous blog post: “To me, developing with Grails always felt like walking on eggs.”

Regression

Some bugs sneak back in again or are even reopened. Note that this is not the same as bugs, bugs, bugs because fixed bugs should be secured by a test.

Leaky abstractions

You have to know the underlying technologies especially Hibernate and Spring to get a foot on the ground. The GORM layer inherits all the complexity from Hibernate.

Slow integration tests

The ramp up time is 45 s on a decent build/development machine and then the first test hasn’t even started.

Uses the Java way of solving problems

Got a problem? There’s a framework for that!

Abandoned or prototype like plugins

Take a look at the list of plugins like Autobase, Flex.

Problems with incremental compiling

Don’t know where the real cause is buried: but using IntelliJ for developing Grails projects results in comments like:
Not working? Have you cleaned, invalidated your caches, rebuilt your project, deleted the .grails directory?

The ugly things

Things which are easily avoided or just a minor issue.

Groovys use of == and equals

Inherited from Java and made even worse: compare two numbers or a String and a GString

Groovys definition for the boolean truth

0, [], “”, null, false are all false

Groovys use of the NullObject and the plus operator

Puzzler: what is null + null ?

Uses unsupported/discontinued technologies

Hibernates SchemaExport comes to mind.

Mix of technology and intention

hasMany, hasOne, belongsTo have not only an intention revealing function but also determine how cascading works and the schema is generated.

Summary, opinionated

Grails has deficits and is bug ridden. But this will be better in the future (hopefully).
When you compare Grails with standard web stacks in the Java world you can gain a lot from it.
So if you want to know if you should use Grails in your next project ask yourself:

  • do you have or want to use Spring and Hibernate?
  • can you live without static typing? (remember: with freedom comes responsibility)
  • are you ready to work around or even fix an issue or bug?
  • is Java your home?

If you can answer all those questions with Yes, then Grails is for you. But beware: no silver bullets!

Fluent code – challenge your compiler

Learn how to leverage the abilities of your compiler to achieve highly readable code in Java (and probably other similar languages).

Making code more readable, that is, easier to read and therefor easier to grasp, has always been an important secondary goal for me when writing code. The primary goal is correctly working software, but immediately after the code works, it enters maintainance mode. Refactoring is a great tool to improve the structure and accessibility of existing code, but it doesn’t necessarily lead to code that is more readable. I’ve even found that there are multiple levels of “easily accessible” code, depending on your experience with different code structures. But that’s another topic for another blog post.

Readable code

Before I can talk about how to create readable code, I have to define what “readable” means to me: I see readable code as code everybody can read (out loud) and directly understand without further reference.

Here’s an example of a little code snippet in Java that follows my definition:

ForeachFile.in(directory).checkIf(IsOlder.than(5).days());

If you replace the parentheses and dots with whitespace, you can read the line fluently and gain a proper idea of what it is doing.

I’ve always found it much easier to write code similar to this example in dynamic languages. In Groovy, Scala or Perl, you are used to invent your own domain specific language dialect that’s much more readable and concise than using the underlying API directly with all the tedious details. But with a bit of practice, Java (and other statically typed languages) are nearly as flexible to reach (or get near) the highest level of readability: code in natural language.

Start with a sentence

The easy way to accomplish the really challenging task of matching computer programming language and naturally spoken language is to pass it on to the compiler. Start with the desired behaviour of a line of code written as a sentence. The compiler will raise all kinds of objections against this form of programming, and all you have to do is to follow the compiler’s complaints, add some special characters and camel casing and then fill out the classes and methods you just planned ahead.

In reality, it will not be as easy as outlined above, but the process stays the same:

  1. Write your desired code, neglecting all compiler errors
  2. Identify method calls, method parameters, class names and other language features as it fits best
  3. Outline the next code you’ll have to write by silencing the compiler with code stubs (use the code generation features of your IDE)
  4. Fill out the (empty) spots you just created, starting with point 1.

Your first attempts might not be as successful as hoped, so you have to backtrack and adjust for perfectly fluent code to a slightly less perfect form, but that’s just reasonable. You still came up with the possibly most readable code you were able to write.

Know when to stop

Although the process seems to be indefinitely repeatable as you descend deeper and deeper in your code (assuming you started with rather high-level code), there will be a fine line when you have to stop the process because the technical aspects of your code will overwhelm every attempt to wrap natural language around it. You probably still have a good amount of perfectly readable code that even non-programmers can grasp at first sight. Just if you dig deeper into its details, the readability will fade.

Your code will be partitioned into two regions: One region is meant to be read, understood and adjusted if requirements change. The other part of the code isn’t as readable and exists mostly to support the first type of code. This is where you still have to be a programmer to make a change. I assume that your partitioning will meander on the border between business requirements and technical implementation.

Observations along the way

My experimentation phase with this kind of programming revealed some insights that mostly other developers made intuitively when exposed to this style in pair programming sessions.

The most interesting revelation was that the names of my classes change: Instead of using nouns, I tend to use verbs in combination with prepositions (like CheckThat, CreateSome or WaitUntil). This is unfamiliar when reading the class name in isolation, but won’t bother you if you read it in the context of the use case.

Which brings me to the next revelation: The resulting code from the abovementioned process seems to be highly focussed on the current use case. It’s not that it isn’t modifiable or inflexible, but it will serve the task at hand in the best way and fall somewhat short for other use cases. It’s in the ability of the developer to refactor the code once additional use cases appear.

Due to the structuring the natural language imposes on the code, refactorings seem to have a “scope” that can verify if the change at hand is really suitable to bring the code forward. It will be very obvious if a refactoring breaks the ruling structure of the code – the readability of your code will degrade.

Another example

Here is another example of readable code written by the process described above, this time copied from an acceptance test:

station.currentPackage().withTypicalContent().send();
WaitUntilPackage.from(stationName).isProcessedWithin(
    Wait.LONGER).asShownOn(center().statusbar());
Wait.SHORT.perform();
assertThatFilesAreStoredInArchive();
assertThatFilesAreStoredOn(ftpSpace, with(exportName));

You can see that it aren’t always the classnames that drive the code, method names are just as important. And you can see the fitting usage of a code squiggle in the last line, a technique I often use to squeeze in the last missing pieces of fluency.

Summary

Writing readable code that can be read and understood by virtually everyone is a tough task. The programming cycle presented in this article uses the compiler’s ability to complain and the feature of modern IDE to create code stubs (named “quick fixes” or alike) to outline naturally readable code and then fill out the gaps in the best attempt. The result will be code that looks like plain english for the most important parts of the code, the translation of the business requirements. The downside is slightly unusual naming and structure in the other parts of your code.

If you have experiences with this approach to readable code, let us know about it.

Grails upgrade – lessons learned

Grails is a great framework for rapidly developing web applications on top the proven java/servlet platform. Especially smaller, short-lived projects can be a real breeze with all the scaffolding, GORM and convention-over-configuration built into grails.

We happen to use it for a quite complex web application project for almost 3 years now. Half a year ago we upgrade from grails 1.0.x to 1.3.4. That makes 3 major versions in one upgrade step and produced obviously a lot of work and many small bugs. I do not want to put the blame on the grails guys here, because most of the stuff was mentioned in the release notes and it was a big step we decided to take when the decision came to continue the project for several years to come.

Our upgrade policy changed due to that experience and we try to stay a lot more current to be able to adapt our software to framework changes more incrementally. Some weeks ago we upgraded from 1.3.4 to 1.3.7 and this experience was not pleasant. Even though we skimmed through the changelogs and release notes and thought the update should be uncritical for us grails behaviour changed in two aspects which broke things for us:

  1. An API-change where GroovyPagesException was changed to GrailsTagException
  2. Behavioural change where no application context and injections are available in functional tests anymore

Item 1 was easy to fix but you need really good testing to spot it before it slips into production. Such subtle API changes should not happen in micro-version updates as that can easily break parts of the system whithout you knowing because of groovy/grails’ dynamic nature. No compiler saves you here.

Item 2 produced some amount of work for us because we build a quite extensive acceptance test suite using services and domain objects to setup the initial environment for each test. Luckily, there is the grails remote control plugin which you can use for things like that.

Lessons learned

  • You should have extensive automated test suites when developing a grails application over a longer period because things can break in unexpected ways without code changes on your side.
  • Try to plan upgrades some time ahead of releases and dedicate time to scanning the release notes and actually performing the upgrade. It may take you significantly longer than the smooth upgrade procedure itself suggests.

The grails team seems to be increasingly aware of backwards compatibility but they still have some way to go. We hope and expect to see fewer unexpected breakages to occur in the future.

The Grails performance switch: flush.mode=commit

Some default configuration options of Grails are not optimal for all projects.

— Disclaimer —
This optimization requires more manual work and is error prone but isn’t this with most (big) performance improvements?
For it to really work you have to structure your code accordingly and flush explicitly.
—
Recently in our performance measurements of a medium sized Grails project we noticed a strange behavior: every time we executed the same query the time it took increased. It started with 40ms and every time it took 1 ms more. The query was simple like Child.findAllByParent(parent)
The first thought: indexes! We looked at the database (a postgresql db) and we had indexes on the parent column.
Next: maybe the session cache got too large. But session.flush() and session.clear() did not solve that problem.
Another post suggested using a HQL query. Changing to

Child.executeQuery("select new Child(c.name, c.parent) from Child c where parent=:parent", [parent: parent])

had no effect.
Finally after countless more attempts we tried:

session.setFlushMode(FlushMode.COMMIT)

And not even the query executed in constant time it was also 10x faster?!
Hmmm…why?
The default flush mode in Grails is set to AUTO
Which means that before every query made the session is flushed. Every query regardless of the classes effected. The problem is known for hibernate but after 4! years it is still unresolved.
So my question here is: why did Grails chose AUTO as default?

Test Framework Classpath Forgery

A lesson learnt when using HttpUnit with all its dependencies. Xerces changed the system behaviour, but with the test classpath only.

Recently, I had an interesting problem using a testing framework with third-party dependencies. When writing integration tests with JUnit against a very small embedded web application (think of the web based management console for your printer as an example), I chose to use HttpUnit as an auxiliary framework to reduce and clarify the test code.

HttpUnit for testing web applications

If you need to test a classic request/response web application, HttpUnit serves its purpose very well. You can write test code concise and to the point. Downloading and integrating HttpUnit is straight-forward, you can immediately get it to work. Here is an example of a test that asserts that there is at least one link on the web application’s main page:

WebConversation web = new WebConversation();
WebResponse response = web.getResponse(fromServer(port));
WebLink[] allLinks = response.getLinks();
assertTrue("No links found on main webpage", ArrayUtil.hasContent(allLinks));

Test failures appear

After this test was written and included into the build, the continuous integration suddenly reported test failures – in the unit tests. I didn’t change any test there and had no need to change the production code, either. So what was causing the test to fail?

The failing unit test class was very old, ensuring the persistence of some data structure to XML and back. The test that actually failed took care of the XML parser behaviour when an empty XML file was read:

public void testReadingEmptyXML() throws IOException {
    try {
        new XMLQueryPersister(new StringReader(XMLQueryPersisterTest.EMPTY_XML), null).loadQueries();
        Assert.fail();
    } catch (ParseException e) {
        Assert.assertEquals("Error on line 1: Premature end of file.", e.getMessage());
    }
}

The assertion that checks the exception message failed, stating that the actual message was now “Error on line -1: Premature end of file.”

Hunting the bug

How can the inclusion of a new integration test have such an impact on the rest of the system? Thanks to continuous integration, the cause for the behaviour change could only lie in the most recent commit. A quick investigation revealed the culprit:

HttpUnit has a third-party dependency on the Xerces xml parser (or another equivalent org.xml.sax parser), see their FAQ for details. When I included the libraries, I accidentally changed the default xml parser for the whole system to Xerces in the version that HttpUnit delivers. This altered the handling of the “premature end of file” case to the new behaviour, causing the test to fail. As these libraries are only included in the classpath when tests are run, the change only happens in the test environment, not in production.

Test classpath versus production classpath

The real issue here isn’t the change in behaviour, this can be taken into account if you have a good test coverage. The issue is different classpaths for test and production environments. If you don’t want to deploy all your test scope libraries (thus making the production classpath similar to the test classpath), you should pay extra attention to what you include in your test classpath. It might alter your system, so that you don’t test the real behaviour anymore.

Resolving the issue

In my case, it was sufficient to remove the Xerces jar from the classpath again. A compliant org.xml.sax parser is already included in the Java core API. It’s the parser that already got used in production and should be used for the tests, too.

Update/Correction: After removing Xerces, HttpUnit stopped working correctly. The quick fix now is to include Xerces in the production classpath and deal with the behaviour changes. I will investigate this issue further and append the outcome as a comment to this blog entry. Update 2: Issue resolved, see comment section for the solution.

This taught me a lesson to always be aware of the dependencies, even if it’s “only” the test scope dependencies.

Summary

Including the Xerces xml parser as a dependency for a testing framework (HttpUnit) changed the behaviour of my system under test, albeit for the tests only. The issue was easily resolved by removing it again, but now I know that testing frameworks have side effects, too.

A tale of scrap metal code – Part III

The third and last part of a series about the analysis of a software product. This part tries to give some rules of thumb on how to avoid failure like in this project.

In the first part of this tale about an examined software project, I described the initial situation and high-level observations about the project. The second part dove into the actual source code and pointed out what’s wrong on this level. This part will summarize everything and give some hints on how to avoid creating scrap metal code.

About the project

If you want to know more about the project, read the first part of this tale. In short, the project looked like a normal Java software, but unfolded into a nightmare, lacking basic requirements like tests, dependency management or continuity.

A summary of what went wrong

In short, the project failed in every respect except being reasonable functional and delivering business value to the customers. I will repeat this sentence soon, but let’s recall the worst parts again. The project had no tests. The project modularization was made redundant by circular dependencies and hardwired paths. No dependency management was in place, neither through the means of a build tool nor by manual means (like jar versions). The code was bloated and overly complex. The application’s data model was a widely distributed network of arbitrary collections with implicit connections via lookup keys. No effort was spent to grasp exception handling or multithreading. The cleverness was rather invested into wildcard usage of java’s reflection API capabilities. And when the cleverness of the developer was challenged, he resorted to code comments instead of making the code more accessible.

How can this be avoided?

First, you need to know exactly what it is you want to avoid. Let me repeat that the project was sold to happily paying customers who gained profit using it. Many software projects fail to deliver this utmost vital aspect of virtually every project. The problem with this project isn’t apparent yet, because it has a presence (and a past). It’s just that it has no future. I want to give some hints how to develop software projects with a future while still delivering business value to the customer.

Avoid the no-future trap

http://www.istockphoto.com/stock-photo-5407438-percent-blocks.phpThe most important thing to make a project future-proof is to restrain yourself from taking shortcuts that pay off now and need to be paid back later. You might want to believe that you don’t need to pay back your technical debts (the official term for these shortcuts) or that they will magically disappear sometimes, but both scenarios are quite unlikely. If your project has any chance to keep being alive over a prolonged amount of time, the technical debts will charge interest.

Of course you can take shortcuts to meet tight deadlines or fit into a small budget. This is called prototyping and it pays off in terms of availability (“time to market”) and scope (“trial version”). Just remember that a prototype isn’t meant for production. You definitely need the extra time and/or budget to fix the intentional shortcomings in the code. You won’t feel the difference right now (hey, it works, what else should it do?), but it will return with compound interest in a few years. The project in this tale was dead after three years. The technical debt had added up beyond being repairable.

Analyzing technical debts

It’s always easy to say that you should “do it right” in the first place. What could the developer for project at hands have done differently to be better off now?

1. Invest in automated tests

When I asked why the project has no tests at all, the developer replied that “it surely would be better to have tests, yet there was no time to write them“. This statement implies that tests take more time to write than they save acting as a guideline and a safety net. And it is probably true for every developer just starting to write tests. You will feel uncomfortable, your tests will be cumbersome and everything will slow down. Until you gain knowledge and experience in writing tests. It is an investment. It will pay off in the future, not right now. If you don’t start now, there will be no future payout. And even better: now your investment, not your debt, will accumulate interest. You might get used to writing tests and start being guided by them. They will mercilessly tell you when your anticipated solution is overly complex. And they will stay around and guard your code long after you forgot about it. Tests are a precaution, not an afterthought.

2. Review and refactor your code

The project has a line count of 80,000 lines of ugly code. I’m fairly confident that it can be reduced to 20,000 lines of code without losing any functionality. The code is written with the lowest possible granularity, with higher concepts lurking everywhere, waiting to be found and exposed. Of course, you cannot write correct, concise and considerate code on your first attempt. This is why you should revisit old code in a recurring manner. If you followed advice number one and brought your tests in place, you can apply every refactoring of the book’s catalog and still be sure that you rather fixed this part instead of breaking it. Constantly reviewing and refactoring your code has the additional advantage of a code base that gets more proficient alongside yourself. There are no “dark regions” (the code to never be read or touched again, because it hurts) if you light them up every now and then. This will additionally slow you down when you start out, but put you on afterburner when you realize that you can rescue any code from rotting by applying the refactoring super-powers that you gained through pratice. It’s an investment again, aiming at midterm return of investment.

3. Refrain from clever solutions

The project of this tale had several aspects that the developer thought were “clever”. The only thing with “clever” is that it’s a swearword in software programming. Remember the clever introduction of wildcard runtime classloading to provide a “plugin mechanism”? Pure poison if you ever wanted your API to be stable and documented, just like a plugin interface should be. Magic numbers throughout your code? Of course you are smart enough to handle this little extra obfuscation. Except when you aren’t. You aren’t sure how exception handling works? Be clever and just “empty catch Exception” everywhere the compiler points you to. In this project, the developer knew this couldn’t be the right solution. Yet, he never reviewed the code when he one day knew how to handle exceptions in a meaningful manner. Let me rest my case by stating that if you write your code as clever as you can handle it, you won’t be able to read it soon, as reading code is harder than writing it.

Summary

Over the course of this tale, you learned a lot about a failed project. In this article, I tried to give you some advice (in the form of three basic rules) on how this failure could probably have been avoided. Of course, the advice isn’t complete. There is much more you could do to improve yourself and your project. Perhaps the best self-training program for developer skills is the Clean Code Developer Initiative (it’s mostly german text yet, so here is an english blog post about it), based upon the book “Clean Code” by Robert C. Martin (Uncle Bob).

Invest in the future of your project and stay clean.

SSL with POCO

A short introduction to using SSL support in POCO C++ libraries.

Admittedly, the topic of this post is very specific but I hope it will still be of some value for some people.The task for today is to setup SSL server and client with POCO framework classes. I will leave out the whole certificate managing issues and just assume that the right files are at hand.

The SSL related part of  the POCO libraries essentially wraps the OpenSSL library into a nice object-oriented interface. When you know OpenSSL, you can instantly relate to classes like Poco::Net::Context, or the …Handler classes (if you replace “handler” with “callback”).

“SSL” stands for Secure Socket Layer, so the first thing to discover is class Poco::Net::SecureServerSocket. As you would expect, this class is derived from Poco::Net::ServerSocket, extending it only with SSL related stuff. And sure enough, some constructors of Poco::Net::ServerSocket take a Poco::Net::ContextPtr as argument.

But why only some constructors? Since there is no setContext method, there must be some other mechanism in place by which SecureServerSockets get their SSL context.

Introducing Poco::Net::SSLManager. From the API docs:

SSLManager is a singleton for holding the default server/client Context and handling callbacks for certificate verification errors and private key passphrases.

Proper initialization of SSLManager is critical.

Aha! So all the constructors of SecureServerSocket that do not take Context pointers simply get it from the SSLManager singleton.

But how to initialize SSLManager?

1. The POCO Way:

If you developed your application with POCO from the ground up there probably exists a sub-class of Poco::Application, and all the configuration is handled by the built-in configuration classes.

With this in place, all you have to do is to add the proper ssl configuration elements:

openSSL.server.privateKeyFile = /path/to/key/file
openSSL.server.certificateFile = /path/to/certificate/file
openSSL.server.verificationMode = none
openSSL.server.verificationDepth = 9
openSSL.server.loadDefaultCAFile = false
openSSL.server.cypherList = ALL:!ADH:!LOW:!EXP:!MD5:@STRENGTH
openSSL.server.privateKeyPassphraseHandler.name = KeyFileHandler
openSSL.server.privateKeyPassphraseHandler.options.password = securePassword
openSSL.server.invalidCertificateHandler = AcceptCertificateHandler

2. Manually:

Depending on which side you are – client or server – you have to call SSLManager::initializeClient or  SSLManager::initializeServer. Both methods take three arguments:

  1. PrivateKeyPassphraseHandler pointer
  2. InvalidCertificateHandler pointer
  3. Context pointer

This is where it becomes a little bit tricky: If you try to instantiate a Context with a privateKey file in order to provide it as argument to the initialize… method, a PrivateKeyPassphraseHandler might be needed. This handler is fetched from the SSLManager singleton – which you are just about to initialize!.

This circular dependency between Context and SSLManager can be overcome e.g. if you call SSLManager::initializeServer first only with a PrivateKeyPassphraseHandler, a InvalidCertificateHandler and null Context pointer. Then instantiate the Context and call SSLManager::initializeServer again.

Now that SSL Manager is initialized we can use Secure… prefixed classes as we would used their non-SSL counterparts. As with SecureServerSocket, other Secure… classes are derieved from corresponding non-secure base classes.

Conclusion: Once you got around the initialization of SSLManager singelton, using SSL POCO classes is very easy and straight forward. Check it out!

A tale of scrap metal code – Part II

The second part of a series about the analysis of a software product. This part investigates the source code and reveals some ugly practices therein.

In the first part of this tale about an examined software project, I described the initial situation and high-level observations about the project. This part will dive into the actual source code and hopefully reveal some insights. The third and last part will summarize everything and give some hints on how to avoid creating scrap metal code.

About the project

If you want to know more about the project, read the first part of this tale. In short, the project looked like a normal Java software, but unfolded into a nightmare, lacking basic requirements like tests, dependency management or continuity.

About the developer

The developer has a job title as a “senior developer”. He developed the whole project alone and wrote every line of code. From the code, you can tell his initial uncertainty, his learning progress, some adventurous experiments and throughout every file, a general uneasiness with the whole situation. The developer actively abandoned the project after three years of steady development. From what I’ve seen, I wouldn’t call him a “senior” developer at all.

About the code

The code didn’t look very repellent at first sight. But everywhere you looked, there was something to add on the “TODO list”. Let me show you our most prominent findings:

Unassigned constructors

The whole code was littered with constructor calls that don’t store the returned new object. What’s the point in constructing another instance of you throw it away in the next moment, without ever using it? After examining these constructors, it became apparent that they only exist to perform side effects each. The new object is registered with the global data model while it’s still under construction. It was the most dreadful application of the Monostate design pattern I’ve ever seen.

Global data model

Did I just mention the global data model? At the end of the investigation, we found that the whole application state lives in numerous public static arrays, collections or maps. These data fields are accessible from everywhere in the application and altered without any protection against concurrent modifications. These global variables were placed anywhere, without necessarily being semantically associated with their enclosing class. A data model in the sense of some objects being tied together to form an instance net with higher-level structures could not be found. Instead, different lookup structures like index-based arrays and key-based maps are associated by shared keys or obscure indices. The whole arrangement of the different data pieces is implicit, you have to parse the code for every usage. Mind you, these fields are globally accessible.

Manual loop unrolling

Some methods had several hundred lines of the exact same method call over and over again. This is what your compiler does when it unrolls your foreach loops. In this code, the compiler didn’t need to optimize. To add some myth, the n-th call usually had a slight deviation from the pattern without any explanation. Whenever something could easily be repeated, the developer pasted it all over the place. Just by winding up the direct repititions again, the code migth shrink by one quarter in length.

Least possible granularity

Just by skimming over the code, you’d discover plenty of opportunities to extract methods, raising the level of abstraction in the code. The developer chose to stick with the least possible granularity, making each non-trivial code a pain to read. The GUI-related classes, using Swing, were so bloated by trivialities that even a simple dialog with two text fields and one button was represented by a massive amount of code. Sadly, the code was clearly written by hand because of all the mistakes and pattern deviations. If the code had to deal with complex data types like dates, the developer always converted them to primitive data types like int, double or long and performed the necessary logic using basic math operators.

“This code is single-threaded, right?”

Despite being a Java Swing application, the code lacked any strategy to deal with multithreading other than ignoring the fact that at least two threads would access the code. We didn’t follow this investigation path down to its probably bitter end, but we wouldn’t be surprised if the GUI would freeze up occasionally.

“Exceptions don’t happen here”

If you would run a poll on the most popular exception handling strategy for this code, it would be the classic “local catch’n’ignore”. The developer dismissed the fact that exceptions might happen and just carried on. If he was forced to catch an exception, the catch block followed immediately and was empty in most cases. Of course, the only caught exception type was the Exception class itself.

“This might be null“

One recurring pattern of the developer was a constructor call, stored in a local variable and immediate null check. Look at this code sample:

try {
    SomeObject object = new SomeObject();
    if (object != null) {
        object.callMethod();
    }
    [...]
} catch (Exception e) {
}

There is no possibility (that I know of) of object being null directly after the constructor call. If an exception is thrown in the constructor, the next lines won’t be executed. This code pattern was so prevalent in the code that it couldn’t be an accidental leftover of previous code. The accompanying effect were random null checks for used variables and return values.

Destabilized dependencies

If there is one thing that’s capable of derailing every code reader, analysis tool and justified guess, it’s wildcard use of Java’s reflection capabilities. The code for this project incorporates several dozens calls to Class.forName(), basically opening up the application for any code you want to dynamically include. The class names result from obscure string manipulation magic or straight from configuration. It’s like the evil brother of dependency injection.

Himalaya indentation

Looking at the indentation depths of the code, this wasn’t the worst I’ve ever seen. But that doesn’t mean it was pleasant. Like in Uncle Bob’s infamous “A crap odyssey”, you could navigate some classes by whitespace landscape. “Scroll down to the fifth crest, the vast valley afterwards contains the detail you want to know”.

Magic numbers

The code was impregnated with obscure numbers (like 9, 17, 23) and even more bizzare textual constants (like “V_TI_LB_GUE_AB”) that just appeared out of nowhere several times. This got so bad that the original author included lengthy comment sections on top of the biggest methods to list the most prominent numbers alongside their meanings. Converting the numbers to named constants would probably dispel the unicorns, as we all know that unicorns solely live on magic numbers(*). Any other explanation escapes my mind.

(*) On a side note, I call overly complex methods with magic numbers “unicorn traps”, as the unicorns will be attracted by the numbers and then inevitably tangled up in the complexity as they try to make their way out of the mess.

Summary

This was the list of the most dreaded findings in the source code. Given enough time, you can fix all of them. But it will be a long and painful process for the developer and an expensive investment for the stakeholder.

To give you an overall impression of the code quality, here is a picture of the project’s CrapMap. The red rectangles represent code areas (methods) that need improvements (the bigger and brighter, the more work it will take). The green areas are the “okay” areas of the project. Do you see the dark red cluster just right the middle? These are nearly a hundred complex methods with subtle differences waiting to be refactored.

Prospectus

In part three, I’ll try to extract some hints from this project on how to avoid similar code bases. Stay tuned.