Tag: programming

My Favorite Pattern

It has become somewhat of an internal meme that I do not like it when programmers use the word “wrapper”. When someone does say it, I usually get a cue from one of the others to start complaining about it. Do not get me wrong, though. I am very much in favor of wrapping things, but with purpose. And my favorite one is the façade.

When simple becomes complex

Many times, APIs start out simple and elegant. This usually works for a while and the API gets used a lot precisely because of its beauty and simplicity. But eventually, a new use case comes along that demands more of the API than it can currently serve. It has to be extended. This usually takes the form of an additional method or function parameter, or an additional function that needs to be called. Using the API now becomes more complex all its users.

Do not underestimate this effect. I have only anecdotal evidence, but in my experience, a lot of unnecessary software complexity can be attributed to this1. The Pareto-Principle applies here: A single use case causes all the users of the previously simple API to deal with new complexity (e.g. 10% of the use cases cause 90% of the complexity in the user-/call-sites).

Façades make it look beautiful

Luckily, it can be dealt with beautifully: using the façade pattern. This pattern abstracts a complex API behind a simple API. The trade-off, of course, is that it is less powerful than the “full API”. In our example though, all of the previous use-cases can keep using the simple API via a façade.

When to apply this

The aforementioned example, extending an API, is a very nice opportunity to apply the façade. Just keep the interface of the old API around, and re-implement it using the new, extended API, which is usually created by modifying the old API’s implementation. Now all the old call-sites can stay the same, yet you can have a more powerful API for those rare cases that need it.

Of course, you can also identify common usage patterns and refactor them using a façade, but that’s usually much harder to do.

What exactly are façades made of?

Façades do not hide the more complex API in the sense that the APIs users are not allowed to use it. Yes, façades make APIs look beautiful, but that is where the metaphor ends. You can still access what is behind the façade. You can even write more façades for the behind. Many APIs have multiple common cases and only very few complex ones.

So… Classes? Functions? Data? Any of those, in fact. Whenever you enable writing something in a simpler way for a common case, you have a façade . Very often, a small function with a simple signature is all the façade you need.

But it makes all the difference.

Now can someone please tell me what that little hook under the c is called?

  1. Façades can, of course, also contribute to creating complexity by growing the codebase and creating ‘variants’. But they rarely do. ↩︎

Serving static files from a JAR with with Jetty

Web applications are often deployed into some kind of web server or web container like Wildfly, Tomcat, Apache or IIS. For single-instance services this can be overkill and serving requests can easily be done in-process without interfacing with some external web container or web server.

A proven and popular framework for an in-process web container and webserver is Eclipse Jetty for Java. For easy deployment and distribution you can build a single application archive containing everything: the static resources, web request handling, all your code and dependencies needed to run your application.

If you package your application that way there is one caveat when trying to serve static resources from this application archive. Let us have a look how to do it with Jetty.

Using the DefaultServlet for static resources on the file system

Jetty comes with a Servlet-Implementation for serving static resources out-of-the-box. It is called DefaultServlet and can be used like below:

Server server = new Server();
ServerConnector connector = new ServerConnector(server);
connector.setHost(listenAddress);
connector.setPort(listenPort);
server.addConnector(connector);
var context = new ServletContextHandler();
context.setContextPath("/");
var defaultServletHolder = context.addServlet(DefaultServlet.class, "/static/*");
defaultServletHolder.setInitParameter("resourceBase", "/var/www/static");
server.setHandler(context);
server.start();

This works great in the case of static resources available somewhere on the filesystem.

But how do we specify where the DefaultServlet can find the resources within our application archive?

Using the DefaultServlet for static in-JAR resources

The only thing that we need to change is the init-parameter called resourceBase from a normal file path to a path in the JAR. What does a path to the files inside a JAR look like and how do we construct it? It took me a while to figure it out, but here is what I came up with and it works perfectly in my use cases:

private String getResourceBase() throws MalformedURLException {
	URL resourceFile = getClass().getClassLoader().getResource("static/existing-file-inside-jar.txt");
    return new URL(Objects.requireNonNull(resourceFile).getProtocol(), resourceFile.getHost(), resourceFile.getPath()
        .substring(0, resourceFile.getPath().lastIndexOf("/")))
        .toString();
}

The method results in a string like jar:file:/path/to/application.jar!/static. Using such a path as the resourceBase for the DefaultServlet allows you to serve all the stuff from the /static directory inside your application (or any other) jar containing the class this method resides in.

A few notes on the code

Why don’t we just hardcode a path after the jar:-protocol? Well, the file path may chance in several scenarios:

  • Running the application on a different platform or operating system
  • Renaming the application archive – it could contain the version number for example…
  • Installing or copying the application archive to a different location on the file system

Using an existing resource and reusing the relevant parts of the URL-specification for the resource base directory solves all these issues because it is computed at runtime.

However, the code assumes that there is at least one resource available in the JAR and that its path is known at compile time.

In newer JDKs like 21 LTS constructing an URL using the constructor is deprecated but I did not bother to rewrite the code to use URI because of time constraints. That is left up to you or a future release…

As always I hope someone finds the code useful and drops a comment.

Where the Wild Boxes Are

When I was a little child, a book that had a big impact on me and my view on the world was “Where the Wild Things Are”. Many years later, it helped me to explain my passion and profession to my grandparents. This blog post tries to give an approach to explain software development to non-technical people.

The first encounters

My first contact with computers was when I was five or six years old in the laboratory of my father. He was a young physicist and worked virtually around the clock in his university’s laboratory. The lab itself was a magical place full of machines and dangerous things like liquid nitrogen canisters. In order to keep me from touching things, he let me play with the only machine that could do no harm: the personal computer on his desk. My interaction with it basically boiled down to moving the cursor on the screen and placing characters into pictures.

When I was eight years old, we got our own family personal computer at home. This was the start of my lifelong passion to teach the machine new tricks. Of course I played every game I could get hold on, but at same time, I wanted to create my own games. By copy-typing code listings from magazines I checked out from the local library, I taught myself to transform my ideas into source code. By trial and error, I expanded my vocabulatory until I could talk to the computer in a nearly fluent fashion.

The apprenticeship

I was sure about my career wish since these days. When my extended family (like aunts and grandparents) asked what I would do once school was finished, I could tell them that I “study something with computers”. It was sufficient as an outlook.

During my studies, it was more important to them that I was studying seriously than what exactly it was that I was studying. They asked about my grades, but not about the content.

The translation gap

Then I started my company and began to earn money with my skills. That’s when the questions about what exactly I was doing emerged. And I learned that the concept of “programming a computer” is not universally understood.

My grandparents weren’t technical people. One grandfather was a railroad worker and had mechanical skills, but couldn’t grok electronics, let alone digital systems. We tried to find a level of simplification of my work that he could imagine and landed at “rapidly pressing buttons in the right order”. In his mind, I was a silent variation of a pianist.

While this is flattering, it lacks the aspect of persistence. A piano falls silent once the button-pressing is done. My computers commence their play long after my typing. A piano player is expected to repeat his “typing”, while my code only needs to be written once and can be copied automatically. The piano player teaches one specific piano how to produce music, my code can teach lots of computers at once how to produce numbers or “data”.

The wild boxes

Data is another concept that is hard to imagine with a mechanical worldview. So I tried another communication approach: the “animal tamer”. Instead of the end result, I focused on the computation process itself. I explained that every computer has its own set of behaviour and can react on incoming information (we used the metaphor of e-mails or “electronic letters”) on its own. The problem is that computers are very dumb and need extensive training to act professional. The training comes in the form of instructional electronic letters (the program code) that the computers read and adapt to.

My job is to write the instruction letters and make the computers read them. This tames the wild box and turns them into domesticated machines that work for us, just like horses or dogs.

To my surprise, this explanation lit up my grandfather’s face: “You tame machines and teach them how to read!” He could understand this process and my role in it. And because machine taming sounds dangerous and important, I earned my wages.

Domesticated boxes

In the book from my childhood, the protagonist Max befriends a group of monsters and gets them to act according to his plan. In my life, I befriend computers and make them act according to my plan. I like the metaphor of “taming” or “domesticating” computers because it highlights the benefits of my work instead of its mechanics.

We build our modern world on billions of domesticated, well-behaved computers. They work for us in exactly the way we told them. But they won’t improve by themselves because we never told them how to learn.

Self-domesticating boxes

Right now, we change our approach of teaching them. Instead of telling them what to do, we try to let them figure it out themselves by trial and error. The beneficial potential is that the machine is not burdened with our limited understanding of the world of digital data. It may be able to expand its vocabulatory until it can interact with its world in a fluent fashion.

Maybe in the future, we need to bargain with our machines so that they work for us. Maybe the next generation of “computer kids” will explain their work to me as “machine mediators”. I’m curious!

Do most language make false promises?

Some years ago I stumbled over this interesting article about C being the most effective of programming language and one making the least false promises. Essentially Damien Katz argues that the simplicity of C and its flaws lead to simple, fast and easy to reason about code.

C is the total package. It is the only language that’s highly productive, extremely fast, has great tooling everywhere, a large community, a highly professional culture, and is truly honest about its tradeoffs.

-Damien Katz about the C Programming language

I am Java developer most of the time but I also have reasonable experience in C, C++, C#, Groovy and Python and some other languages to a lesser extent. Damien’s article really made me think for quite some time about the languages I have been using. I think he is right in many aspects and has really good points about the tools and communities around the languages.

After quite some thought I do not completely agree with him.

My take on C

At a time I really liked the simplicity of C. I wrote gtk2hack in my spare time as an exercise and definitely see interoperability and a quick “build, run, debug”-cycle as big wins for C. On the other hand I think while it has a place in hardware and systems programming many other applications have completely different requirements.

  • A standardized ABI means nothing to me if I am writing a service with a REST/JSON interface or a standalone GUI application.
  • Portability means nothing to me if the target system(s) are well defined and/or covered by the runtime of choice.
  • Startup times mean nothing to me if the system is only started once every few months and development is still fast because of hot-code replacement or other means.
  • etc.

But I am really missing more powerful abstractions and better error handling or ressource management features. Data structures and memory management are a lot more painful than in other languages. And this is not (only) about garbage collection!

Especially C++ is making big steps in the right direction in the last few years. Each new standard release provides additional features making code more readable and less error prone. With zero cost abstractions at the core of language evolution and the secondary aim of ease of use I really like what will come to C++ in the future. And it has a very professional community, too.

Aims for the C++11 effort:

  • Make C++ a better language for systems programming and library building
  • Make C++ easier to teach and learn

-Bjarne Stroustup, A Tour of C++

What we can learn from C

Instead of looking down at C and pointing at its flaws we should look at its strengths and our own weaknesses/flaws. All languages and environments I have used to date have their own set of annoyances and gotchas.

Java people should try building simple things and having a keen eye on dependencies especially because the eco system is so rich and crowded. Also take care of ressource management – the garbage collector is only half the deal.

Scala and C++ people should take a look at ABI stability and interoperability in general. Their compile times and “build, run, debug”-cycle has much room for improvement to say the least.

C# may look at simplicity instead of wildly adding new features creating a language without opinion. A plethora of ways implementing the same stuff. Either you ban features or you have to know them all to understand code in a larger project.

Conclusion

My personal answer to the title of this blog: Yes, they make false promises. But they have a lot to offer, too.

So do not settle with the status quo of your language environment or code style of choice. Try to maintain an objective perspective and be aware of the weaknesses of the tools you are using. Most platforms improve over time and sometimes you have to re-evaluate your opinion regarding some technology.

I prefer C++ to C for some time now and did not look back yet. But I also constantly try different languages, platforms and frameworks and try to maintain a balanced view. There are often good reasons to choose one over the other for a particular project.

 

Getting better at programming without coding

Almost two decades ago one of the programming books was published that had a big impact on my thinking as a software engineer: the pragmatic programmer. Most of the tips and practices are still fundamental to my work. If you haven’t read it, give it a try.
Over the years I refined some practices and began to get a renewed focus on additional topics. One of the most important topics of the original tips and of my profession is to care and think about my craft.
In this post I collected a list of tips and practices which helped and still help me in my daily work.

Think about production

Since I develop software to be used, thinking early about the production environment is key.

Deploy as early as possible
Deployment should be a non event. Create an automatic deployment process to keep it that way and deploy as early as possible to remove the risk from unpleasant surprises.

Master should always be deployable
Whether you use master or another branch, you need a branch which could always be deployed without risk.

Self containment
Package (as many as possible of) your dependencies into your deployment. Keep the surprises of missing repositories or dependencies to a minimum of none.

Use real data in development
Real data has characteristics, gaps and inconsistencies you cannot imagine. During development use real data to experience problems before they get into production.

No data loss
Deploying should not result in a loss of data. Your application should shutdown gracefully. Often deployment deletes the directory or uses a fresh place. No files or state in memory should be used as persistence by the application. Applications should be stateless processes.

Rollback
If anything goes wrong or the new deployed application has a serious bug you need to revert it to the last version. Make rollback a requirement.

No user interruption
Users work with your application. Even if they do not lose data or their current work when you deploy, they do not like surprises.

Separate one off tasks
Software should be running and available to the user. Do not delay startup with one off admin tasks like migration, cache warm-up or search index creation. Make your application start in seconds.

Manage your runs
Problems, performance degradation and bugs should be visible. Monitor your key metrics, log important things and detect problems in the application’s data. Make it easy to combine, search and graph your recordings.

Make it easy to reproduce
When a bug occurs or your user has a problem, you need to follow the steps how the system arrived at its current state. Store their actions so that they can be easily replayed.

Think about users

Software is used by people. In order to craft successful applications I have to consider what these people need.

No requirements, just jobs
Users use the software to get stuff done. Features and requirements confuse solutions with problems. Understand in what situation the user is and what he needs to get to his goal. This is the job you need to support.

Work with the user
In order to help the user with your software I need to relate to his situation. Observing, listening, talking to and working along him helps you see his struggles and where software can help.

Speak their language
Users think and speak in their domain. Not in the domain of software. When you want to help them, you and the user interface of your software needs to speak like a user, not like a software.

Value does not come from effort
The most important things your software does are not the ones which need the most effort. Your users value things which help them the most. Find these.

Think about modeling

A model is at the core of your software. Every system has a state. How you divide and manage this state is crucial to evolving and understanding your creation.

Use the language of the domain
In your core you model concepts from the user’s domain. Name them accordingly and reasoning about them and with the users is easier.

Everything has one purpose
Divide your model by the purpose of its parts.

Separate read from write
You won’t get the model right from the start. It is easier to evolve the model if read and write operations have their own model. You can even have different read models for different use cases. (see also CQRS and Turning the database inside out)

Different parts evolve at different speeds
Not all parts of a model are equal. Some stand still, some change frequently. Some are specified, about some others you learn step by step. Some need to be constant, some need to be experimented with. Separating parts by its changing speed will help you deal with change.

Favor immutability
State is hard. State is needed. Isolating state helps you understand a running system. Isolating state helps you remove coupling.

Keep it small
Reasoning about a large system is complicated. Keep effects at bay and models small. Separating and isolating things gives you a chance to overview the whole system.

Think about approaches

Getting to all this is a journey.

When thinking use all three dimensions
Constraining yourself to a computer screen for thinking deprives you of one of your best thinking tools: spatial reasoning. Use whiteboards, walls, paper and more to remove the boundaries from your thoughts.

Crazy 8
Usually you think in your old ways. Getting out of your (mental) box is not easy. Crazy 8 is a method to create 8 solutions (sketches for UI) in a very short time frame.

Suspend judgement
As a programmer you are fast to assess proposals and solutions. Don’t do that. Learn to suspend your judgement. Some good ideas are not so obvious, you will kill them with your judgement.

Get out
Thinking long and hard about a problem can put you into blindfold mode. After stating the problem, get out. Take a walk. Do not actively think or talk about the problem. This simulates the “shower effect”: getting the best ideas when you do not actively think about the problem.

Assume nothing
Assumptions bear risks. They can make your project fail. Approach your project with what is certain. Choose your direction to explore and find your assumptions. Each assumption is an obstacle, an question that needs an answer. Ask your users. Design hypotheses and experiments to proof them. (see From agile to UX for a detailed approach)

Pre-mortem
Another way to find blind spots in your thinking is to frame for failure. Construct a scenario in which your project is failed. Then reason about what made it fail. Where are your biggest risks? (see How to map your fears for details)

MVA – Minimum, valuable action
Every step, every experiment should be as lightweight as possible. Do not craft a beautiful prototype if a sketch would suffice. Choose the most efficient method to get further to your goal.

Put it into a time box
When you need to experiment, constrain it. Define a time in which you want to have an answer. You do not need to go the whole way to get an impression.

Be(a)ware of Laziness

Let’s assume we have a simple JavaScript “class” called Module. Each instance of the class has a name, a start() method and a stop() method to manage its lifecycle:

function Module(name) {
    this.name = name;
    console.log("Creating " + this.name);
}
Module.prototype.start = function() {
    console.log("Starting " + this.name);
};
Module.prototype.stop = function() {
    console.log("Stopping " + this.name);
};

We want to create a couple of instances with the names “a”, “b” and “c”. At the beginning of the program we want to start each module, and at the end of the program we want to stop each module. For the creation of the instances we use a map() function call on the names array:

var names = ["a", "b", "c"];
var modules = names.map(function(name) {
    return new Module(name);
});
modules.forEach(function(module) {
    module.start();
});
// do something
modules.forEach(function(module) {
    module.stop();
});

The output is as intended:

Creating a
Creating b
Creating c
Starting a
Starting b
Starting c
Stopping a
Stopping b
Stopping c

Now we want to port this code to C#. The definition of the class is straight-forward:

class Module
{
    private readonly String name;

    public Module(string name)
    {
        this.name = name;
        Console.WriteLine("Creating " + name);
    }

    public void Start()
    {
        Console.WriteLine("Starting " + name);
    }

    public void Stop()
    {
        Console.WriteLine("Stopping " + name);
    }
}

The map() function is called Select() in .NET:

var names = new List<string>{"a", "b", "c"};
var modules = names.Select(
                 name => new Module(name));

foreach (var module in modules)
{
    module.Start();
}

foreach (var module in modules)
{
    module.Stop();
}

But when we run this program, we get a completely different output:

Creating a
Starting a
Creating b
Starting b
Creating c
Starting c
Creating a
Stopping a
Creating b
Stopping b
Creating c
Stopping c

Each module is created twice, and the creation calls are interleaved with the start() and stop() calls.

What has happened?

The answer is that .NET’s Select() method does lazy evaluation. It does not return a new list with the mapped elements. It returns an IEnumerable instead, which evaluates each mapping operation only when needed. This is a very useful concept. It allows for the chaining of multiple operations without creating an intermediate list each time. It also allows for operations on infinite sequences.

But in our case it’s not what we want. The stopped instances are not the same as the started instances.

How can we fix it?

By appending a .ToList() call after the .Select() call:

var modules = names.Select(
        name => new Module(name)).ToList();

Now the IEnumerable gets evaluated and collected into a list before the assignment to the modules variable.

So be aware of whether your programming language or framework uses lazy or eager evaluation for functional collection operations to avoid running into subtle bugs. Other examples of tools based on the concept of lazy evaluation are the Java stream API or the Haskell programming language. Some languages support both, for example Ruby since version 2.0:

range.collect { |x| x*x }
range.lazy.collect { |x| x*x }