Tag: classpath

Test Framework Classpath Forgery

A lesson learnt when using HttpUnit with all its dependencies. Xerces changed the system behaviour, but with the test classpath only.

Recently, I had an interesting problem using a testing framework with third-party dependencies. When writing integration tests with JUnit against a very small embedded web application (think of the web based management console for your printer as an example), I chose to use HttpUnit as an auxiliary framework to reduce and clarify the test code.

HttpUnit for testing web applications

If you need to test a classic request/response web application, HttpUnit serves its purpose very well. You can write test code concise and to the point. Downloading and integrating HttpUnit is straight-forward, you can immediately get it to work. Here is an example of a test that asserts that there is at least one link on the web application’s main page:

WebConversation web = new WebConversation();
WebResponse response = web.getResponse(fromServer(port));
WebLink[] allLinks = response.getLinks();
assertTrue("No links found on main webpage", ArrayUtil.hasContent(allLinks));

Test failures appear

After this test was written and included into the build, the continuous integration suddenly reported test failures – in the unit tests. I didn’t change any test there and had no need to change the production code, either. So what was causing the test to fail?

The failing unit test class was very old, ensuring the persistence of some data structure to XML and back. The test that actually failed took care of the XML parser behaviour when an empty XML file was read:

public void testReadingEmptyXML() throws IOException {
    try {
        new XMLQueryPersister(new StringReader(XMLQueryPersisterTest.EMPTY_XML), null).loadQueries();
        Assert.fail();
    } catch (ParseException e) {
        Assert.assertEquals("Error on line 1: Premature end of file.", e.getMessage());
    }
}

The assertion that checks the exception message failed, stating that the actual message was now “Error on line -1: Premature end of file.”

Hunting the bug

How can the inclusion of a new integration test have such an impact on the rest of the system? Thanks to continuous integration, the cause for the behaviour change could only lie in the most recent commit. A quick investigation revealed the culprit:

HttpUnit has a third-party dependency on the Xerces xml parser (or another equivalent org.xml.sax parser), see their FAQ for details. When I included the libraries, I accidentally changed the default xml parser for the whole system to Xerces in the version that HttpUnit delivers. This altered the handling of the “premature end of file” case to the new behaviour, causing the test to fail. As these libraries are only included in the classpath when tests are run, the change only happens in the test environment, not in production.

Test classpath versus production classpath

The real issue here isn’t the change in behaviour, this can be taken into account if you have a good test coverage. The issue is different classpaths for test and production environments. If you don’t want to deploy all your test scope libraries (thus making the production classpath similar to the test classpath), you should pay extra attention to what you include in your test classpath. It might alter your system, so that you don’t test the real behaviour anymore.

Resolving the issue

In my case, it was sufficient to remove the Xerces jar from the classpath again. A compliant org.xml.sax parser is already included in the Java core API. It’s the parser that already got used in production and should be used for the tests, too.

Update/Correction: After removing Xerces, HttpUnit stopped working correctly. The quick fix now is to include Xerces in the production classpath and deal with the behaviour changes. I will investigate this issue further and append the outcome as a comment to this blog entry. Update 2: Issue resolved, see comment section for the solution.

This taught me a lesson to always be aware of the dependencies, even if it’s “only” the test scope dependencies.

Summary

Including the Xerces xml parser as a dependency for a testing framework (HttpUnit) changed the behaviour of my system under test, albeit for the tests only. The issue was easily resolved by removing it again, but now I know that testing frameworks have side effects, too.

Lightweight dependency management

Managing project dependencies without maven or ant ivy, using a custom ant task to ensure classpath orthogonality.

Java’s classpath is a powerful concept – when used appropriate. As your project grows larger in terms of code and people, it gets harder to ensure that your classpath is correct. A great danger arises from JAR files containing different versions of the same resource. You might end up running different code than you think, leading to strange effects. If you build your classpath using wildcards, you can’t even control the order your JAR files are loaded.

Managing dependencies

To avoid the issues mentioned above, you need to manage your project dependencies. It’s a common practice to implement the build process of the project using maven or ant ivy. Both tools provide dependency mangement by declaration. But at a high cost. Especially maven has received some malice lately, criticizing its steep learning curve and complexity.

Scratching the biggest itch

We decided to try a different approach to dependency management, tackling only our biggest concern: The duplication of classpath resources. We take care of the scope of a third-party library, put required JARs in the repository (to us, third party binary artifacts are part of the project source) and update manually. The one thing we cannot assure manually is that every resource is unique. Sometimes, the same class is included in different JARs, as it seems to be common practice among java web frameworks.

Ant to the rescue

Thus, I wrote a custom ant task that, given the classpath, checks for duplicate entries. If it finds one, it lists the culprits and optionally aborts the build process. Included in our continuous integration system, it gets run every time somebody performs a change. You can’t forget to delete an old version of a library or check in the same library twice without breaking the build now.

Our ClasspathCollisionCheckTask

I provide this task here, without any warranty. The source code is included in the JAR alongside the classes, if you want to know what it does exactly.

Assuming you already know how to use custom tasks within an ant build script, here’s only a short usage description.

Import the custom task:

<taskdef
    name="check.collision"
    classname="com.schneide.internal.anttask.ClasspathCollisionCheckTask"
    classpath="${customtasks.library.directory}/schneidetasks.jar"
/>

Next, use it on your classpath:

<check.collision verbose="true" failOnCollision="true">
    <path>
        <fileset dir="${classpath.library.directory}">
            <include name="**/*.jar"/>
        </fileset>
        <fileset dir="${internal.library.directory}">
            <include name="**/*.jar"/>
        </fileset>
    </path>
</check.collision>

The task scans the whole path you give it and reports any collision it detects. You will see the warnings in your build log.

If the failOnCollision parameter is set to true (optional, defaults to false), the build will abort after a collision. If you want to have debug information, set the verbose parameter to true (optional, defaults to false).

Conclusion

If you manage your project dependencies manually, you might find our custom ant task useful. If you use maven or ant ivy, you already have this functionality in your build process.

Feedback

I’m very interested in hearing your opinion on the task or about your way of handling dependencies. Leave us a comment.