Category: Software Development

Simple C++11 – Part I – Unit Structure

C++ has long had the stigma of an overlay complex and unproductive language. Lately, with the advent of C++11, things have brightened a bit, but there are still a lot of misconceptions about the language. I think this is mostly because C++ was taught in a wrong way. This series aims to show my, hopefully somewhat simpler, way of using C++11.

Since it is typically the first thing I do when starting a new project, I will start with how I am setting up a new compile unit, e.g. a header and compile unit pair.

Note that I will try not to focus on a specific C++11 paradigm, such as object-oriented or imperative. This structure seems to work well for all kinds of paradigms. But without much further ado, here’s the header file for my imaginary “MyUnit” unit:

MyUnit.hpp

#pragma once

#include <vector>
#include "MyStuff.hpp"

namespace MyModule { namespace MyUnit {

/** Does something only a good bar could.
*/
std::vector<float> bar(int fooCount);

/** Foo is an integral part of any program.
    Be sure to call it frequently.
*/
void foo(MyStuff::BestType somethingGood);

}}

I prefer the .hpp file ending for headers. While I’m perfectly fine with .h, I think it is helpful to differentiate pure C headers from C++ headers.

#pragma once

I’m using #pragma once here instead of include guards. It is not an official part of the standard, but all the big compilers (Visual C++, g++ and clang) support it, making it a de-facto standard. Unlike include guards, you only have to add only one line, which says exactly what you want to achieve with it. You do not have to find a unique identifier for your include guard that will most certainly break if you rename the file/unit. It’s more readable, more resilient to change and easier to set up.

Namespaces

I like to have all the contents of a unit in a single namespace. The actual structure of the namespaces – i.e. per unit or per module or something else entirely depends on the specifics of the project, but filling more than one namespace is a guarantee for chaos. It’s usually a sign that the unit should be broken up into smaller pieces. An exception to this would be the infamous “detail” namespace, as seen in many of the Boost libraries. In that case, the namespace is not used to structure the API, but to explicitly omit things from the API that have to be visible for technical reasons.

Documentation

Documentation goes into the header, not into the implementation. The header describes the API, not only to the compiler, but also to humans. It is by no means an implementation detail, but part of the seam that isolates it from the rest of the code. Note that this part of the documentation concerns the API contract only, never the implementation. That part goes into the .cpp file.

But now to the implementation file:

MyUnit.cpp

#include "MyUnit.hpp"

#include "CoolFunctionality.hpp"

using namespace MyModule;
using namespace MyUnit;

namespace {

int helperFunction(float rhs)
{
  /* ... */
}

}// namespace

std::vector<float> MyUnit::bar(int fooCount)
{
  /* ... */
}

void MyUnit::foo(MyStuff::BestType somethingGood)
{
  /* ... */
}

Own #include first

The only rule I have for includes is that the unit’s own include is always the first. This is to test whether the header is self-sufficient, i.e. that it will compile without being in the context of other headers or, even worse, code from an implementation file. Some people like to order the rest of their includes according to their “origin”, e.g. sections for system headers or library headers. I think imposing any extra order here is not needed. If anything, I prefer not waste time sorting include directives and just append an include when I need it.

Using namespace

I choose using-directives of my unit’s namespaces over explicitly accessing the namespaces each time. Unlike the headers, the implementation file lives in a locally defined context. Therefore, it is not a problem to use a very specific view onto the unit. In fact, it would be a problem to be overly generic. The same argument also holds for other “local” modules that this unit is only using, as long as there are no collisions. I avoid using namespaces from external libraries to mark the library boundary (such as std, boost etc.).

Unnamed namespace

The unnamed namespace contains all the implementation helpers specific to this unit. It is quite common for this to contain a lot of the “meat” of an actual unit, while the unit’s visible functions merely wrap and canonize the functionality implemented here. I try to keep only one unnamed namespace in each file, to have a clear separation of what is supposed to be visible to the outside – and what is not.

Visible implementation

The implementation of the visible API of the module is the most obvious part of the .cpp file. For consistency reasons, the order of the functions should be the same as in the header.

I’d advice against implementing in a file wide open namespace. That means balancing an unnecessary pair of parenthesis over the whole implementation file.  Also, you can not only define functions and types, but also declare them – this leads to a function further down in the implementation to see a different namespace than one before it.

Conclusion

This concludes the first part. I’ve played with the thought of using a 3-piece setup instead, extending the header/implementation with a unit-test file, but have not gathered any sharable experience yet. This setup, however, has worked for me for a long time and with many different projects. Have you had similar – or completely different – setups that worked for you? Do tell!

Synchronizing asynchronous calls in JavaScript

In Node.js all I/O performing operations like HTTP requests, file access etc. are designed to be non-blocking. The functions for these operations usually take a callback function argument as last parameter, which will be called once the operation is finished. The asynchronous nature of these operations often requires special means of synchronization, as exemplified in this post.

Let’s assume we want to download a set of files from a list of URLs:

var urls = [
    'http://example.org/file1',
    'http://example.org/file2',
    'http://example.org/file3',
];

If we were to use a classic, synchronous blocking function for the download operation the implementation might look like this:

urls.forEach(function(url) {
    downloadSync(url);
    console.log('Downloaded ' + url);
});
console.log('Downloads complete.')

Output:

Downloaded http://example.org/file1
Downloaded http://example.org/file2
Downloaded http://example.org/file3
Downloads complete.

The program loops through the list of URLs and downloads one file after the other. Each subsequent download is only started after the previous download has finished. No two downloads are active at the same time. Finally, the program prints a message indicating that all downloads are complete.

Now we want to embrace the asynchronous programming style of Node.js. We have a different function, downloadAsync, which is a non-blocking asynchronous function. The function uses the callback convention of Node.js to let the programmer handle the completion of the asynchronous operation:

urls.forEach(function(url) {
    downloadAsync(url, function() {
        console.log('Downloaded ' + url);
    });
});
console.log('Downloads complete.')

Output:

Downloads complete.
Downloaded http://example.org/file2
Downloaded http://example.org/file3
Downloaded http://example.org/file1

The download operations are now started at the same time and finish in a different order, depending on how long it takes each file to download. The quickest download finishes first, the slowest last. The total time for all downloads to finish is probably shorter than before, because slower connections do not make faster connections wait for them to finish.

However, there is one problem with this code: the message “Downloads complete.” is shown before the downloads are actually completed, which is obviously not the intended behavior. The reason why this happens is because the control flow immediately continues after the forEach loop has started all the downloads.

The ‘async’ module

What we need to correct this behavior is a way to wait for all asynchronous download operations started by the loop to finish before we print the “Downloads complete” message.

The async module for JavaScript provides various synchronization mechanisms for exactly these kinds of situations. In our case we’re going to use async.each:

var async = require('async');

async.each(urls, function(url, callback) {
    downloadAsync(url, function() {
        console.log('Downloaded ' + url);
        callback();
    });
}, function done() {
    console.log('Downloads complete.')
});

The async.each function is similar to JavaScript’s forEach function. However, the function called for each iteration has an additional callback parameter, which must be called when each iteration is considered to be completed. The third parameter to async.each is also a callback function. This function is called after all iterations reported their completion. This is where we place the output of the “Downloads complete” message.

Conclusion

The async module provides a rich toolset for the synchronization of asynchronous operations in JavaScript. Go check it out!

Multi-Page TIFFs with C++

If you are dealing with high-speed cameras or other imaging equipment capable of producing many images in a short time you may find it handy to put many images into a single file. There are several reasons to do so:

  • Dealing with thousands of files in a single directory or spreading them over a directory hierarchy may be slow and cumbersome depending on the tools.
  • Storing many images together may communicate better that they belong together, e.g. to the same scan.
  • Handling and transmitting fewer files is often easier than juggling with many.

TIFF is a wide-spread lossless image format capable of handling many individual images in a single file. Many image viewers and image manipulation tools are able to work with multi-page TIFF files so you are quite flexible in working with such files.

But how do you produce these files from your programs? I found some solutions with different strengths and weaknesses:

Using Magick++

Magick++ is the C++ API for ImageMagick – a powerful image manipulation library. If you can hold all the images for one file in memory the code is easy and straightforward:

#include &amp;lt;string&amp;gt;
#include &amp;lt;Magick++.h&amp;gt;

class TiffWriter
{
public:
    TiffWriter(std::string filename);
    TiffWriter(const TiffWriter&amp;amp;) = delete;
    TiffWriter&amp;amp; operator=(const TiffWriter&amp;amp;) = delete;
    ~TiffWriter();

    void write(const unsigned char* buffer, int width, int height);

private:
    std::vector&amp;lt;Magick::Image&amp;gt; imageList;
    std::string filename;
};

TiffWriter::TiffWriter(std::string filename) : filename(filename) {}

// for example for a 8 bit gray image buffer
void TiffWriter::write(const unsigned char* buffer, int width, int height)
{
    Magick::Blob gray8Blob(buffer, width * height);
    Magick::Geometry size(width, height);
    Magick::Image gray8Image(gray8Blob, size, 8, "GRAY");
    imageList.push_back(gray8Image);
}

TiffWriter::~TiffWriter()
{
    Magick::writeImages(imageList.begin(), imageList.end(), filename);
}



The caveat is that you need to  hold all your images in memory before writing it to the file on disk. I did not manage to add and persist images on the fly to disk.


In our environment it was absolutely necessary to do so because of the amount of data and the I/O required to persist all image in time. So I had to implement a slightly more low-level solution using libtiff and its C API.


Using libtiff


#include &amp;lt;string&amp;gt;
#include &amp;lt;tiffio.h&amp;gt;

class TiffWriter
{
public:
    TiffWriter(std::string filename, bool multiPage);
    TiffWriter(const TiffWriter&amp;amp;) = delete;
    TiffWriter&amp;amp; operator=(const TiffWriter&amp;amp;) = delete;
    ~TiffWriter();

    void write(const unsigned char* buffer, int width, int height);

private:
    TIFF* tiff;
    bool multiPage;
    unsigned int page;
};

TiffWriter::TiffWriter(std::string filename, bool multiPage) : page(0), multiPage(multiPage)
{
    tiff = TIFFOpen(filename.c_str(), "w");
}

void TiffWriter::write(const unsigned char* buffer, int width, int height)
{
    if (multiPage) {
        /*
         * I seriously don't know if this is supposed to be supported by the format,
         * but it's the only we way can write the page number without knowing the
         * final number of pages in advance.
         */
        TIFFSetField(tiff, TIFFTAG_PAGENUMBER, page, page);
        TIFFSetField(tiff, TIFFTAG_SUBFILETYPE, FILETYPE_PAGE);
    }
    TIFFSetField(tiff, TIFFTAG_PLANARCONFIG, PLANARCONFIG_CONTIG);
    TIFFSetField(tiff, TIFFTAG_IMAGEWIDTH, width);
    TIFFSetField(tiff, TIFFTAG_IMAGELENGTH, height);
    TIFFSetField(tiff, TIFFTAG_SAMPLEFORMAT, SAMPLEFORMAT_UINT);
    TIFFSetField(tiff, TIFFTAG_ROWSPERSTRIP, TIFFDefaultStripSize(tiff, (unsigned int) - 1));

    unsigned int samples_per_pixel = 1;
    unsigned int bits_per_sample = 8;
    TIFFSetField(tiff, TIFFTAG_BITSPERSAMPLE, bits_per_sample);
    TIFFSetField(tiff, TIFFTAG_SAMPLESPERPIXEL, samples_per_pixel);

    std::size_t stride = width;
    for (unsigned int y = 0; y &amp;lt; height; ++y) {
        TIFFWriteScanline(tiff, buffer + y * stride, y, 0);
    }

    TIFFWriteDirectory(tiff);
    page++;
}

TiffWriter::~TiffWriter()
{
    TIFFClose(tiff);
}



Note that line 14 is needed if you do not know the number of images to store in the file in advance!

Meet my Expectations!

A while ago I came across a particulary irritating piece of code in a somewhat harmlessly looking mathematical vector class. C++’s rare feature of operator overloading makes it a good fit for multi-dimensional calculations, so vector classes are common and I had already seen quite a few of them in my career. It looked something like this:

template <typename T>
class vec2
{
public:
  /* A few member functions.. */
  bool operator==(vec2 const& rhs) const;

  T x;
  T y;
};

Not many surprises here, except that maybe the operator==() should be a free-function instead. Whether the data members of the class are an array or named individually is often a point of difference between vector implementations. Both certainly have their merits. But I digress…
What really threw me off was the implementation of the operator==(). How would you implement it? Intuitively, I would have expected pretty much this code:

template <typename T>
bool vec2<T>::operator==(vec2 const& rhs) const
{
  return x==rhs.x && y==rhs.y;
}

However, what I found instead was this:

template <typename T>
bool vec2<T>::operator==(vec2 const& rhs) const
{
  if (x!=rhs.x || y!=rhs.y)
  {
    return false;
  }

  return true;
}

What is wrong with this code?

Think about that for a moment! Can you swiftly verify whether this boolean logic is correct? You actually need to apply De Morgan’s laws to get to the expression from the first implementation!
This code was not technically wrong. In fact, for all its technical purposes, it was working fine. And it seems functionally identical to the first version! Still, I think it is wrong on at least two levels.

Different relations

Firstly, it bases its equality on the inequality of its contained type, T. I found this quite surprising, so this already violated the POLA for me. I immediately asked myself: Why did the author choose to implement this based on operator!=(), and not on operator==()? After all, supplying equality for relations is common in templated C++, while inequality is inferred. In a way, this is more intuitive. Inequality already has the negation in its name, while equality is something “original”! Not only that, but why base the equality on a different relation of the contained type instead of the same? This can actually be a problem when the vector is instantiated on a type that supplies operator==(), but not operator!=() – thought that would be equally surprising. It turned out that the vector was only used on built-in types, so those particular concerns were futile. At least, until it is later used with a custom type.

Too many negations

Secondly, there’s the case of immediately returning a boolean after a condition. This alone is often considered a code-smell. It could be argued that this is more readable, but I don’t want to argue in favor of pure brevity. I want to argue in favor of clarity! In this case, that construct is basically used to negate the boolean expression, further obscuring the result of the whole function.
So basically, the function does a double negation (not un-equal) to express a positive concept (equal). And negations are a big source of errors and often lead to confusion.

Conclusion

You need to make sure to make the code as simple and clear as possible and avoids any surprises, especially when dealing with the relatively unconstrained context of C++ templates.  In other words, you need to make sure to meet the expectations of the naive reader as well as possible!

Transferring commits via Git bundles

Sometimes you want to send (e.g. by e-mail) a set of new Git commits to someone else who has the same repository at an older state, without transferring the whole repository and without sharing a common remote repository.

One feature that might come to your mind are Git patches. Patches, however, don’t work when there are branches and merge commits in the commit history: git format-patch creates patches for the commits across the various branches in the order of their commit times and doesn’t create patches for merge commits.

Git bundles

The solution to the problem are Git bundles. Git bundles contain a partial excerpt of a Git repository in a single file.

This is how to create a bundle, including branches, merge commits and tags:

$ git bundle create my.bundle <base commit>..HEAD --branches --tags

<base commit> must be replaced with the last commit (i.e. commit hash or tag), which was included in the old state of the repository.

A Git bundle can be imported into a repository via git pull:

$ git pull /path/to/my.bundle

Get it up and running

West Side-project story
West Side-project story

I have seen quite a few projects that spent a ton of calendar/developer time and bugdet in building components and frameworks instead of getting something running. Running in this sense does not mean being able to show an application started from a developers IDE on her development notebook. With running I mean versioned artifacts deployed on some sort of staging infrastructure the client/customer has access to. Let me elaborate on that:

Walking skeleton

I really like the notion of the walking skeleton coined by Steve Freeman and Nat Pryce in “Growing Object-Oriented Software, Guided by Tests“. While I do not want to emphasize TDD and/or automated end-to-end testing I see great benefits in producing a walking skeleton that touches all important parts of a system and is working with a minimal set of functionality. For me that means that all of the following elements are in place, albeit in a primitve way which can be refined on the way:

  • The code is hosted in a source code repository accessible to the project members
  • A build system is chosen and able to produce a runnable artifact
  • A continuous integration server (CI) is triggered on changes in the repository and produces the runnable artifact
  • The artifact is easily installable on the target machines and/or installed on a staging system that resembles the target system as closely as reasonable
  • If there are components they talk to another using minimal requests and stubbed replies that can be refined over time

I usually aim for that walking skeleton in the first few hours into a project after the base technologies and requirements allow starting with coding. That may take up to several days when the system is more complex but it should not take weeks. Connect different parts of the system as early as possible even if responses are minimal, hardcoded or “wrong”. It shows that the parts are able to communicate and mismatches will become visible either someway along the build process or at least when running application. Why should you choose such an approach?

Benefits

  • Most people I know are better at evolving and improving existing stuff than at creating new stuff in empty space in a focused and efficient way. If you have a skeleton of the system it is much easier to talk about the interfaces and responsibilites of the different parts of the system.
  • You get to define APIs which you can evolve over time instead of specifing the complete API and experience implementation and mismatch problems much later when the components need to be integrated.
  • Similarily it is much more difficult to package a complex system after it is finished than to package a simple minimal system and evolve all aspects – building, implemenation and packaging/deployment, maintainance – when necessary.
  • You see if things may work like expected or if there is some inherent problem in the whole design much earlier. Essentially you evolve your proof-of-concept into something with real value for your clients.
  • As soon as your system provides some value you can deploy the working stuff long before the whole project is finished.
  • It is much easier to discuss a working system than to reason about a system not yet existing.
  • You are eating your own dogfood from early on and can address pain points in development, user experience, deployment and running the application.
  • You have something to show more or less right from the beginning and progress will be visible throughout the project.
  • No “Works on my machine!” syndrome.

Potential Problems

Of course there is the challenge of continuously refactoring and extending existing stuff. Later on data migration or migration of configuration may be additional tasks. But hey, you are skilled, agile developers that embrace the idea of changing requirements and the ability to move fast. You have to pay attention not to accumulate too much technical debt as it will slow you down and hurt you in the long run.

Summary

Running systems providing value are what your clients often care about the most. If you can something like that early on communication with your stakeholders tends to be more relaxed as they have better possibilities of steering in the right direction and they steadily see progress. Running software should be the primary goal.

Thinking in immutability

The way I learned programming is dictated by objects and states. Besides advantages and on going efforts in the industry I couldn’t help but thinking: immutability is nice. I can use it in some cases and keep it quietly stored in the corner. But it didn’t remain silent.

The way I learned programming is dictated by objects and states. According to my thinking data is packed into objects which are later modified to reflect the changes over time. State and modification are a central modelling technique. For me programming and OOP in particular resolved around this common theme. Mutating objects pervade my thinking even beyond the code into the database and even the architecture of the whole system.
Besides advantages and on going efforts in the industry I couldn’t help but thinking: immutability is nice. I can use it in some cases and keep it quietly stored in the corner.
But it didn’t remain silent.
So I asked myself: How do you construct programs that build upon immutability? How do you (mostly) avoid mutable objects? How do you think in immutability?
The first step was to unlearn. No updates. No modifications. Read, create, copy. That’s about it. No more CRUD only CR. No more SQL updates, only inserts.

Events and logs

To illustrate I use a simple example. Creating, moving, translating and deleting a point. In the traditional OO way it looks like this:

Point p = new Point(40, 30);
p.translateXBy(5);
p.moveTo(10, 20);
p.delete();

Or using SQL might be something like this (omitting primary keys and where clauses here):

insert into points (x, y) values (40, 30)
update points p set p.x = p.x + 5
update points p set p.x = 10, p.y = 20
delete from points

In our memory (or database if we use one) every line updates our point:

Point p = new Point(40, 30); // p = {x: 40, y: 30}
p.translateXBy(5); // p = {x: 45, y: 30}
p.moveTo(10, 20); // p = {x: 10, y: 20}
p.delete(); // p = ?

But what if we do not store the results of the operations but the operations themselves? The events.
Imagine your state changes as a series of events. Just imagine.

new PointCreated(40, 30); // pointEvents = [{created[x: 40, y:30]}]
new PointTranslatedXBy(5); // pointEvents = [{created[x: 40, y:30]}, {translated[x: 5]}]
new PointMovedTo(10, 20); // pointEvents = [{created[x: 40, y:30]}, {translated[x: 5]}, {moved:[x: 10, y:20]}]
new PointDeleted(); // pointEvents = [{created[x: 40, y:30]}, {translated[x: 5]}, {moved:[x: 10, y:20]}, {deleted}]

Even in the database we would just use inserts, no more updates and no more deletes. The events are stored in a log (ironically the database does this the same way). A log is a fully ordered, append only queue. Once we use and store events we have some extras besides immutability: an audit trail, an undo stack, recovering, …
We could externalize the event stream in a message queue and could monitor it, replay it to reproduce bugs, distribute it. The possibilities are endless.

But. That’s all nice and fine. I have one more question: what’s the current state? A user should see the current state and other parts of the system also (not mentioning that I – coming from a mutable state kind of thinking – would also feel better seeing it).

So what’s the current state?

All events applied in order.

OK. But isn’t this expensive doing this all the time?

Yes!

Here another concept from databases helps us: materialized views. We can easily translate in our mind between the new immutable event driven way and the old in place update way. It is just the same data in different representations (if we only are interested in the current state). If we store the current state as a materialized view (or cache) besides the event log we can have both.
Every part of the program which needs the current state gets an immutable copy of it. If this part needs to know when something changes, it can observe the events and act accordingly. This way mutability is pushed to the borders, to the parts where the current state is shown (like the UI layer).

Quantities in C++ and User Defined Literals

Some weeks ago one of my colleagues wrote about the use and implementation of physical quantities in C#. If you are writing an application in the technical or scientific domain chances are high that you should adhere to his advice and use a suitable representation of physical quantities instead of plain primitive values. Good news is that you can easily port/implement quantities to modern C++ or use existing libraries like Boost.Units.

With C++11 you can go one step further adding the so called User-defined literals. This feature allows definition of suffices for integer, floating-point, character and string literals to produce objects of the desired (quantity) type. While there is nothing wrong with using the multiplication operator to produce quantity instances user-defined literals provide just a little bit more syntactic sugar:

// Your quantity classes...
class Angle;

// operators for user-defined literals
constexpr Angle operator "" _deg(long double deg)
{
    return deg * degrees;
}

constexpr Angle operator "" _deg(unsigned long long int deg)
{
    return deg * degrees;
}

constexpr Angle operator "" _rad(long double rad)
{
    return (rad * 180 / M_PI) * degrees;
}

// add more if needed

This allows you to write code like:

Angle rightAngle = 90_deg;
Angle halfCircle = 3.141_rad;
Angle fullCircle = 4 * 90_deg;

In many cases this looks a tad simpler and cleaner than using the multiplication operator in conjunction with a unit especially in more complex formulas. There are a few things about quantities and user-defined literals in C++ I find noteworthy:

  • These literals are only supported for the built-in literal types. If exact calculation and better than floating-point precision is needed, raw literals (instead of the explained cooked) and decimal libraries have to be used. For raw literals you have to parse the characters of the literal yourself.
  • User-defined literals need to be prefixed with _ to avoid namespace clashes with current and future standard library literals. There are for example some nice literals for durations in the <chrono>-date and time standard library.
  • If you implement your literal operators as constexpr they will be evaluated at compile time meaning slightly increased compile times and zero runtime overhead.

For some more in-depth discussion of user-defined literals have a look at the blog series from Andrzej Krzemieński.

 

What’s your time, database?

Time is a difficult subject. Especially time zones and daylight saving time. Adding layers makes things worse. Ask your database.

Time is a difficult subject. Especially time zones and daylight saving time. Sounds easy? Well, take a look.
Adding layers in software development complicates the issue and every layer has its own view of time. Let’s start with an example: we write a simple application which stores time based data in a SQL database, e.g. Oracle. The table has a column named ‘at’. Since we don’t want to mess around with timezones, we use a column type without timezone information, in Oracle this would be ‘Date’ if we do not need milliseconds and ‘Timestamp’ if we need them. In Java with plain JDBC we can extract it with a call to ‘getTimestamp’:

Date timestamp = resultSet.getTimestamp("at");

The problem is now we have a timestamp in our local timezone. Where is it converted? Oracle itself has two timezone settings: for the database and for the session. We can query them with:

select DBTIMEZONE from dual;

and

select SESSIONTIMEZONE from dual;

First Oracle uses the time zone set in the session, then the database one. The results from those queries are interesting though: some return a named timezone like ‘Europe/Berlin’, the other return an offset ‘+01:00’. Here a first subtle detail is important: the named timezone uses the offset and the daylight saving time from the respective timezone, the offset setting only uses the offset and no daylight saving. So ‘+01:00’ would just add 1 hour to UTC regardless of the date.
In our example changing these two settings does not change our time conversion. The timezone settings are for another column type: timestamp with (local) timezone.
Going up one layer the JDBC API reveals an interesting tidbit:

Timestamp getTimestamp(int columnIndex)
throws SQLException

Retrieves the value of the designated column in the current row of this ResultSet object as a java.sql.Timestamp object in the Java programming language.

Sounds about right, but wait there’s another method:

Timestamp getTimestamp(int columnIndex,
Calendar cal)
throws SQLException


Retrieves the value of the designated column in the current row of this ResultSet object as a java.sql.Timestamp object in the Java programming language. This method uses the given calendar to construct an appropriate millisecond value for the timestamp if the underlying database does not store timezone information.

Just as in Oracle we can use a named timezone or an offset:

Date timestamp = resultSet.getTimestamp("at", Calendar.getInstance(TimeZone.getTimeZone("GMT+1:00")));

This way we have control over what and how the time is extracted from the database. The next time you work with time based information take a close look. And if you work with Java use Joda Time.

Keep your ovens clean

If you have build dependencies that require the build system to be altered, like registered DLLs, there is a workaround that might save you from snowflaking all your machines.

Let’s assume for a moment that you are a baker, producing different types of pastries in your small bakery. The production process is always the same: prepare the dough, put it in the oven, wait some time and retrieve the most delicious buns or bread. If we can abstract the real baking process to these steps, it’s the same as with software: prepare the sourcecode, put it in the compiler, wait some time and retrieve the most delicious binary or executable. There is only one difference: The oven of the baker is a self-contained, closed system, while our compilers require a distinct system setup around them in order to produce anything edible. The oven is independent from the kitchen around it, the compiler is depedent on the environment. To finish the analogy, what would a baker say if he can’t bake bread in his oven unless he nurtures a certain type of yeast in his kitchen?

A most unpleasant case

While developing a platform dependent application recently, we met a most unpleasant case of build dependency on a third-party library. It was an old dynamic link library (DLL) that requires registering in the windows registry. There was no other way than to register the DLL using the regsrv32 utility. If you didn’t do this, the build process would abort with an error stating unmet dependencies. If you ran the resulting program on a machine without registered DLL, it would crash with a runtime error complaining about the missing registry entry. And by the way, there are two totally independent regsrv32 utilities on a 64-bit windows system, one for 32-bit and one for 64-bit registrations. No, the name of the latter one isn’t regsvr64, that would be way too easy.

We accepted the fact that you need to prepare your system if you want to run the program, but we quarreled a lot with the nuisance that you need to alter your system just to build the software. This process of alteration is called snowflaking in the DevOp mentality and it’s not a desired activity. We would need to alter every build machine in our continuous integration cluster that comes into contact with the project. And we would need to de-snowflake them again afterwards, because this kind of tinkering adds up to inscrutable side-effects very fast.

A practicable workaround

We found a way around the abovementioned snowflaking for our build servers. It’s not a solution, it’s only a workaround, as it solves the immediate problem but produces some lesser problems on the way. Let’s look at what we did.

At first, our situation could be described with this module diagram:

dependency1We couldn’t modify the problematic DLL itself, it was a given binary. But we could wrap it in our own DLL. Wrapping less pleasant things into something you can control is a proven technique even in baking, by the way. We now had a system layout that looks like this:

dependency2Nothing gained so far, just that we now have a layer outside our system that can provide the functionality of the DLL and is actually under our control. The wrapper really does nothing on its own but to forward each call to the DLL. To profit from this indirection, we need to introduce another module, like this:

dependency3The second module provides the same interface as the first, but does nothing, not even forwarding anywhere. It’s a complete stub, just there to be uncomplicated during the build process. The goal is to build the system using this “empty” DLL and then replace it with the “problematic” DLL afterwards. The only question is: how do we build the problematic DLL? Here’s the workaround part of the solution: We actually had to compile the problematic DLL on a snowflaked system and add it to the project repository. Good thing our target system’s specification is known, so we only need to do this for one platform. Because we are reasonably sure that the DLL interface will not change over time (it had every opportunity in the last ten years and didn’t use it), we can assume that the interface of our two wrapper DLLs also won’t change. So it’s not too problematic to check in a precompiled binary that needs to satisfy an interface that’s reproduced with every build cycle. Still, we need to keep an eye on the method signatures of our two wrapper DLLs. If one of them changes, the modification needs to be replicated on the other wrapper, too. It’s a classic duplication.

When we balanced the duplication in the interfaces of the two wrapper DLLs against the snowflaking of every CI and developer machine, we found our aversion against snow outweighing the other negative aspects. Your mileage may vary.

Conclusion

We kept our build ovens clean by introducing a wrapping layer around the problematic depedency and then using the benefits of indirection by switching to a non-problematic stub during the build cycle. The technique is very old, but still use- and powerful.