Category: Year

A neat trick for finding dates

One of my fundamental driving forces for designing a suitable user experience is the motto of “no scrolling”. It is not a hard principle in the sense of Bret Victor’s “Inventing on Principle”, but it is motivation enough to always look for context-aware ordering of lists and tables.

One of the cornerstones of such “scrolling-free” or “scrolling-reduced” applications is a possibility for the user to define a context of work. You might call it an ordinary search field, but it is highly interactive (immediate feedback, as mandated by Bret Victor) and doesn’t lead to a special result view. It just reduces the amount of data that is presented to the user. The idea behind the context field is that the user has some idea about the piece of work she wants to edit. So, if for example she remembers that the customer of an order was named “Miller”, she would type “mil” in the context field and have the list of orders reduced to just the “Millers” and “Camilles” (the latter contains “mil” in the middle).

This works fine for text-based information, but less so for numerical data. Most users typically remember text like names better than numbers like values or phone numbers, so it fits the natural inclination of humans. And then there is a form of data that is remembered easily and used for orientation, but presented as numbers: dates.

If you store a date in a persistent storage, it is probably stored as a number, several numbers or a short piece of text like “2024-12-24”. To make a date searchable, the textual representation of “2024-12-24” is a good start, but we can do better with a simple trick:

Instead of just using one textual representation for the search index (or whatever search functionality you use), you can append several representations of the same date at once:

“2024-12-24 Tuesday, December 24, 2024”

Here, we’ve used a code like this:

String searchtext = xmas.toString() + " " + xmas.format(DateTimeFormatter.ofLocalizedDate(FormatStyle.FULL));

This enables your user to restrict to a context of “dec” for “December” or even “tue” for “Tuesday”.

This search-extended representation of the date is never shown on screen, so it doesn’t have to be readable or brief. It should contain the concepts of managing dates that your users will facilitate during their normal interaction with the data (not with your application!). So it might even be useful to add several more representations like a relative period (“in 5 months, 16 days”) and maybe special date names (“christmas, xmas”). It might be useful to cut down on filler words like “months” or “days”, because they are included in virtually every date you’ll search. So the relative period might come down to “5 five, 16 sixteen”. If your search allows for multiple texts that all need to be present (which I would encourage because in my experience, that’s how people triangulate: “It was Miller and at the end of the year”), you might add the filler words again, because it allows for a context string like “5 mo” with matches with “in 5 months” (and “5 months ago”, but that’s another topic).

In a nutshell, my trick is to craft the textual representation of data for the search (not the visualisation!) in accordance to the navigation patterns of my users. If they can rely on the data being focussed effectively, they won’t miss the scrollbars.

Regular expressions in JavaScript

In one of our applications, users can maintain info button texts themselves. For this purpose, they can insert the desired info button text in a text field when editing. The end user then sees the text as a HTML element.

Now, for better structuring, the customer wants to make lists inside the text field. So there was a need to frame lines beginning with a hyphen with the <li></li> HTML tags.

I used JavaScript to realize this issue. This was my first use of regular expressions in JavaScript, so I had to learn their language-specific specials. In the following article, I explain the general syntax and my solution.

General syntax

For the replacement, you can either specify a string to search for or a regular expression. To indicate that it is a regular expression, the expression is enclosed in slashes.

let searchString = "Test";
let searchRegex = /Test/;

It is also possible to put individual parts of the regular expression in brackets and then use them in the replacement part with $1, $2, etc.

let hello = "Hello Tom";
let simpleBye = hello.replace(/Hello/, "Bye");    
//Bye Tom
let bye = hello.replace(/Hello (.*)/, "Bye $1!"); 
//Bye Tom!

In general, with replace, the first match is replaced. With replaceAll, all occurrences are replaced. But these rules just work for searching strings. With regular expressions, modifiers decide if all matches were searched and replaced. To find and replace all of them, you must add modifiers to the expression.

Modifiers

Modifiers are placed at the end of a regular expression and define how the search is performed. In the following, I present just a few of the modifiers.

The modifier i is used for case-insensitive searching. 

let hello = "hello Tom";
let notFound = hello.replaceAll(/Hello/, "Bye");
//hello Tom
let found= hello.replaceAll(/Hello/i, "Bye");
//Bye Tom

To find all occurrences, independent of whether replace or replaceAll is called, the modifier g must be set.

let hello = "Hello Tom, Hello Anna";
let first = hello.replaceAll(/Hello/, "Bye");
//Bye Tom, Hello Anna
let replaceAll = hello.replaceAll(/Hello/g, "Bye");
//Bye Tom, Bye Anna
let replace = hello.replace(/Hello/g, "Bye");
//Bye Tom, Bye Anna

Another modifier can be used for searching in multi-line texts. Normally, the characters ^ and $ are for the start and end of the text. With the modifier m, the characters also match at the start and end of the line.

let hello = `Hello Tom,
hello Anna,
hello Paul`;
let byeAtBegin = hello.replaceAll(/^Hello/gi, "Bye");     
//Bye Tom, 
//hello Anna,
//hello Paul
let byeAtLineBegin = hello.replaceAll(/^Hello/gim, "Bye");     
//Bye Tom, 
//Bye Anna,
//Bye Paul

Solution

With this toolkit, I can now convert the hyphens into HTML <li></li>. I also remove the line breaks at the end because, in real code, they will be replaced with <br/> in the next step, and I do not want empty lines between the list points.

let infoText = `This is an important field. You can input:
- right: At the right side
- left: At the left side`;
let htmlInfo = infoText.replaceAll(/^-(.*)\n/gm, "<li>$1</li>");
//This is an important field. You can input:
//<li>right: At the right side</li><li>left: At the left side</li>

If you are familiar with the syntax and possibilities of JavaScript, it offers good functions, such as taking over parts of the regular expression.

How to get honest UX Feedback from the Technically Adept User? – Part 1: the problem.

After some consideration, I decided to type this title indeed with the Question Mark at its end, because it is an ongoing process, not a particular conclusion. (… and yes, this is a common theme with everything UX).

We have a wide variety of possible customers and usually an even wider variety of possible users. Sometimes, the mission statement is quite well defined, and sometimes, we just start from the acknowledgement that our customer has some pain point and some trust in that we might be able to help.

Of course, we always carefully evaluate whether we believe our uncertainties to stay manageable or rather reiterate the uncertainties / must-have requirements / minimal viable product with the potential customer, upfront. And if in these cases, we find common ground for a collaboration, it is absolutely crucial to always keep the hand on the steering wheel, always reconsidering what it is the user wants.

From very early on, this affects the aspect of User Experience, which is not something one can apply at the end of the project, some lip gloss, a little treat if everything went well before. Wherever possible, it has to be ingrained in the backbone of the software because sooner or later you could risk driving the project into limbo.

UX Limbo is, when your product is good enough so that a potential user will never openly complain about certain design choices, but still too flawed so they will never actually engage in your software, just because… there’s no flow. No dopamine. They won’t tell you – it’s just difficult.

Enter the “Technically Adept User”. If you have a project in which the end users are actually experts in the domain itselves, this is next-level difficult. Here, the problem is not that the user has a underdeveloped mental model, but that they have a time-tested one which has real value for them. But this might differ from the one that you actually implement; and

  • it might be that you “just” need to listen more accurately
  • it might be that you learned something about the problem that it new to them
  • it might be that there are more than one Technically Adept Users, and they do not notice that they hold incompatible models in their head
  • or they do notice, but due to their different roles they try to one-up each other
  • etc.

So this is the problem. If you play your cards well, you might be able to solve a problem in a way that has never been done before, the customers will feel their burdens taken away, replaced by nothing but overwhelming bliss and (…you get the point) – but it doesn’t take much in perceived imperfection, and the users will start doing mistakes, feel unsatisfied, stupid even, never understanding why you “cannot implement the easiest ideas” (most likely your fault), always pretending, always seeming obsessed, wasting their money, just not “getting” it.

And worse is, you cannot even separate yourself from the product that easily. You can always start a UX Session, like A/B Testing, with the Non-Technically-Adept User, let them tell you their emotions, difficulties, misunderstandings and due to the large gap between their mental model and yours, communication is not that hard. But with too similar mental models, they will always think A if you present B. If they don’t understand why something is implemented as B, they can’t stop their thoughts in rationalizing, maybe even nodding their heads, but what you really would need is the open discussion how a common-ground solution C might look.

They might invite you to a meeting and do not even see the points you are seeing, they might consider their problems solved from minute one – and spend the meeting talking about several possible steps ahead.

The Technically Adept User is a blessing in that their knowledge can take the onus in understing the real problem away from you a bit – but is also a challenge because they need to invest more energy in understanding your differences of understanding. There is no universal solution in how to make them.

I am writing this blog post to keep my own thoughts rolling on that topic. There must be some ways in communicating this gap. It should be done in a way that neither lets the user feel dumb, insulted, their pride taken away; but also in a way that they know that you can be both: knowledgeful about their problem AND flexible in iterating through different solutions, trying what works until something sticks. Time and patience are an issue in this, too.

If you have any suggestions or insights to share, please feel free to do so. I have some ideas in mind but will continue with Part 2. Let’s see what we can learn from this 🙂

My conan 2 Consumer Workflow

A great many things changed in the transition from conan 1.x to conan 2.x. For me, as an application-developer first, the main thing was how I consume packages. The two IDEs I use the most in C++ are Visual Studio and CLion, so I needed a good workflow with those. For Visual Studio, I am using its CMake integration, otherwise known as “folder mode”, which lets you directly open a project with a CMakeLists.txt file in it, instead of generating a solution and opening that. The deciding factor for me is that that uses Ninja as a build tool instead of MSBuild, which often is a lot faster. I have had projects with 3.5x build-time speed ups. As an added bonus, CLion supports very much the same workflow, which reduces friction when switching between platforms.

Visual Studio

First, we’re going to need some local profiles. I typically treat them as ‘build configurations’, with one profile for debug and release on each platform. I put them under version control with the project. A good starting point to create them is conan profile detect, which guesses your environment. To create a profile to a file, go to your project folder and use something like:

conan profile detect --name ./windows_release

Note the ./ in the name, which will instruct conan to create a profile in the current working directory instead of in your user settings. For me, this generates the following profile:

[settings]
arch=x86_64
build_type=Release
compiler=msvc
compiler.cppstd=14
compiler.runtime=dynamic
compiler.version=194
os=Windows

Conan will warn you, that this is only a guess and you should make sure that the values work for you. I usually bump up the compiler.cppstd to at least 20, but the really important change is to change the CMake generator to Ninja, after which the profile should look something like this:

[settings]
arch=x86_64
build_type=Release
compiler=msvc
compiler.cppstd=20
compiler.runtime=dynamic
compiler.version=194
os=Windows

[conf]
tools.cmake.cmaketoolchain:generator=Ninja

Copy and edit the build_type to create a corresponding profile for debug builds.

While conanfile.txt still works for specifying your dependencies, I now recommend directly using conanfile.py from the get go, as some options like overriding dependencies are now exclusive to it. Here’s an example installing the popular logging library spdlog:

from conan import ConanFile
from conan.tools.cmake import cmake_layout


class ProjectRecipe(ConanFile):
    settings = "os", "compiler", "build_type", "arch"
    generators = "CMakeToolchain", "CMakeDeps"

    def requirements(self):
        self.requires("spdlog/1.14.1")

    def layout(self):
        cmake_layout(self)

Note that I am using cmake_layout to setup the folder structure, which will make conan put the files it generates in build/Release for the windows_release profile we created.

Now it is time to install the dependencies using conan install. Make sure you have a clean project before this, e.g. there are no other build/config folders like build/, out/ and .vs/. Specifically, do not open the project in Visual Studio before doing that, as it will create another build setup. You already need the CMakeLists.txt at this point, but it can be empty. For completeness, here’s one that works with the conanfile.py from above:

cmake_minimum_required(VERSION 3.28)
project(ConanExample)

find_package(spdlog CONFIG REQUIRED)

add_executable(conan_example
  main.cpp
)

target_link_libraries(conan_example
  spdlog::spdlog
)

Run this in your project folder:

conan install . -pr:a ./windows_release

This will install the dependencies and even tell you what to put in your CMakeLists.txt to use them. More importantly for the Visual Studio integration, it will create a CMakeUserPresets.json file that will allow Visual Studio to find the prepared build folder once you open the project. If there is no CMakeLists.txt when you call conan install, this file will not be created! Note that you generally do not want this file under version control.

Now that this is setup, you can finally open the project in Visual Studio. You should see a configuration named “conan-release” already available and CMake should run without errors. After this point, you can let conan add new configurations and Visual Studio should automatically pick them up through the CMake user presets.

CLion

The process is essentially the same for CLion, except that the profile will probably look different, depending on the platform. Switching the generator to Ninja is not as essential, but I still like to do it for the speed advantages.

Again, make sure you let conan setup the initial build folders and CMakeUserPresets.json and not the IDE. CLion will then pick them up and work with them like Visual Studio does.

Additional thoughts

I like to create additional script files that I use to setup/update the dependencies. For example, in windows, I create a conan_install.bat file like this:

@echo Installing debug dependencies
conan install . -pr:a conan/windows_debug --build=missing %*
@if %errorlevel% neq 0 exit /b %errorlevel%

@echo Installing release dependencies
conan install . -pr:a conan/windows_release --build=missing %*
@if %errorlevel% neq 0 exit /b %errorlevel%

Have you used other workflows successfully in these or different environments? Let me know about them!

Time Intervals in SQL

SQL not only supports date and timestamp data types but also time intervals.

An interval in SQL represents a period of time, such as days, hours, minutes, or seconds. SQL supports intervals in two main types: INTERVAL YEAR TO MONTH and INTERVAL DAY TO SECOND.

The two types

YEAR TO MONTH is used to represent a span of years and months. Examples include calculating ages, durations of contracts, or project timelines.

DAY TO SECOND is used to represent a span of days, hours, minutes, and seconds. Examples include scheduling events, logging activities, and tracking precise durations.

-- Represents 1 year and 6 months
INTERVAL '1-6' YEAR TO MONTH

-- Represents 5 days, 12 hours, 30 minutes, and 45 seconds
INTERVAL '5 12:30:45' DAY TO SECOND

This distinction may seem arbitrary at first, but it aligns with real-world use cases. An interval of “2 months” is clearly different from “60 days,” even though they might seem similar. The former is handled as a “YEAR TO MONTH” interval, and the latter as a “DAY TO SECOND” interval, making it clear how operations should be performed. Storing only the relevant units also reduces the amount of space required. For instance, storing a “YEAR TO MONTH” interval doesn’t need to allocate space for hours or seconds, thus saving storage.

Despite its name a “DAY TO SECOND” interval can also include fractional seconds:

INTERVAL '1 12:30:45.123' DAY TO SECOND

There are also shorter forms of interval literals if you only want to use one or several units of the interval:

For “YEAR TO MONTH”:

INTERVAL '3' YEAR   -- 3 years
INTERVAL '5' MONTH  -- 5 months

For “DAY TO SECOND”:

INTERVAL '2' DAY                -- 2 days
INTERVAL '5:30' HOUR TO MINUTE  -- 5 hours and 30 minutes
INTERVAL '15' MINUTE            -- 15 minutes
INTERVAL '30' SECOND            -- 30 seconds
INTERVAL '45.678' SECOND        -- 45.678 seconds

Note that the syntax always uses the singular forms of the unit words.

The two types can be used as data types for table columns:

CREATE TABLE interval_examples (
  id NUMBER,
  a INTERVAL YEAR TO MONTH,
  b INTERVAL DAY TO SECOND
);

Calculations with intervals

Intervals can be added to or subtracted from dates to calculate future or past dates.

-- Adding 10 days to the current date
CURRENT_DATE + INTERVAL '10' DAY

-- Subtracting 3 months from a specific date
DATE '2024-06-14' - INTERVAL '3' MONTH

It’s important to understand that intervals of type YEAR TO MONTH do not have a fixed duration. The number of days added to CURRENT_TIMESTAMP when you add an INTERVAL ‘1’ MONTH depends on the current month and year, as different months have different numbers of days.

Intervals can also be compared to each other with the usual operators such as =, <, <=, >, >=, <>.

Combining both types

Suppose you need to add an interval of 1 year, 2 months, 3 days, 4 hours, 5 minutes, and 6 seconds to the current timestamp. In this case you need to combine intervals of the two types:

SELECT (CURRENT_TIMESTAMP + INTERVAL '1-2' YEAR TO MONTH) + INTERVAL '3 04:05:06' DAY TO SECOND AS result;

Notes on PostgreSQL

PostgreSQL supports the above standard interval syntax, but it also supports a unified syntax for intervals with the following syntax:

INTERVAL '1 year 2 months'
INTERVAL '10 days 12 hours 30 minutes 45.123 seconds'
INTERVAL '1 year 2 months 3 days 4 hours 5 minutes 6 seconds'

In fact, in PostgreSQL, YEAR TO MONTH and DAY TO SECOND are not a distinct data type on their own; they are specializations of the broader INTERVAL data type.

It also supports multiplication and division of intervals:

-- Results in an interval of 6 days
SELECT INTERVAL '2 days' * 3 AS multiplied_interval;

-- Results in an interval of 2 hours
SELECT INTERVAL '3 hours' / 1.5 AS divided_interval;

Updating Grails: From 5 to 6

We have a long history of maintaining quite a large grails application since Grails 1.0. Over the first few major versions upgrading was a real pain.

The situation changed dramatically after Grails 3 as you can see in our former blog posts and and the upgrade from 3 to 4. Going from 4 to 5 was so smooth that I did not even dedicate a blog post to it.

A few weeks ago we decided to upgrade from version 5 to 6 and here is a short summary of our experiences. Things fortunately went quite smooth again:

The changes

The new major version 6 contains mostly dependency upgrades and official support for Java 17 which is probably the biggest selling point.

Some other minor things to note are without any particular order:

  • The logback configuration file name has changed from logback.groovy to logback-config.groovy
  • The pathing-jar is already setup for you, so you can remove the directive from your build files if you had it in
  • The build uses the standard gradle-application plugin allowing some more simplifications in your build files and infrastructure

Other noteworthy news

Object Computing stepped down as the steward of the grails framework and informed the community in an open letter. While there is still the grails foundation and the open source community we can expect the development and changes to slow down.

Whether this results in negative developer experience remains to be seen.

Conclusion

Using and maintaining a Grails application developed to a rather smooth ride and only poorly maintained plugins hurt your experience. The framework and its foundation have been pretty solid for some time now.

Regarding new projects you certainly have to evaluate if using grails is the best option. For an full stack framework the answer maybe yes, but if you only need a powerful API backend lighter and more modern frameworks like micronaut or javalin may be a better choice.

End each business process with a report

There is the internet saying of “pictures or it didn’t happen”. In the business world, you can translate it to “report or it didn’t happen”. Let me explain.

The concept of a business process is important for this blog post. My impression is that the concept is underrepresented in the business world. I can recall a time where I wasn’t familiar with it. Work occurred and had to be done. No need to make it more complicated than that. Well, it won’t really get more complicated with a business process, just more defined.

A business process is a series of tasks that are directed to a “business objective” or some desirable goal. Everything you do in the context of professional work is probably a task in a business process. The difference to “just work” is that the process is defined in a repeatable manner. There is a description of the necessary steps to do it again. Let’s take a bugfix in a software as an example. The actual content of the bug is different for every iteration of the process. But the basic steps are always the same:

  • A bug report is written by a customer
  • The bug is verified by a developer
  • A test case is written that reveals the bug’s minimal cause
  • The code is changed to fix the bug
  • The test case is changed to protect against the bug
  • The changeset is merged into the upstream code
  • The bug report is marked as fixed
  • The bugfix is released

This might be the rudimentary description of this business process. Note that while all the details of verification effort and code changes are omitted, the process can be repeated on this level of detail many times without variation.

In our company, we describe our business processes as “text with media content” in our wiki. The basic structure of each process consists of four parts that each hold a number of steps:

  • E (Event) – Something happened that requires an iteration of the process
  • A (Actions) – A series of things we need to do to reach the objective, described in varied detail
  • R (Result) – The stop condition for the process. What have we achieved when we perform the actions successfully?
  • B (Report) – Who do we need to inform about the process iteration?

The name for the last part might look strange to you: B for report is only explainable if you translate it into german: “Bericht” means report and starts with a B.

Let’s project these four parts to the example above:

The bug report by the customer is the event. Then follow five steps that are actions (verification, test case, code change, test change, upstream merge). The next step can be interpreted as the report. If you close the bug report ticket, that is the notification that things just got better. The problem here is that the result (the bugfix is released) occurs after the report. This often leads to irritation on the side of the customer that still has the problem even if the ticket is marked as “resolved”.

But this blog post is not about the unfortunate process design that puts the report before the result. It is about the report itself. In my experience, this crucial part of the process is often missing from the description or willfully omitted by the process operator. And that is a problem that can and should be fixed.

A report doesn’t need to be a piece of paper or an e-mail, though it often is. Let me give you two examples of other forms of reports:

  • In hotels, the toilet paper is often folded in a certain way. It indicates that somebody has cleaned this room and it wasn’t used since.
  • The dishwasher in our company is equipped with a new cleaning tab as soon as it was emptied. This indicates that it can be loaded with dirty dishes again. The machine will never close the tab compartment by itself, so if it is closed, it was loaded by a human (who put the clean dishes away beforehands, hopefully).

Without such a report, a business process like the room cleaning or the dishwasher emptying might have occurred, but everybody has to check again on their own.

In the digital world, we often try to automate the reporting by sending out e-mails to ticket reporters and the like. That’s fine. The report step doesn’t need to create extra effort, but it needs to happen.

If you can “think in processes”, you can ask yourself the question “who do I need to inform at the end of this process?”. Who has an interest in the result? Let them know that it is available now! You can’t reach everybody with an interest directly? Think of a way to leave a clue like the folded toilet paper.

In our company, most of our processes are accompanied by a digital ticket, so the resolution of the ticket often triggers the necessary report e-mails. But even in this situation, there are many processes left that require explicit thought and effort to inform the right people.

What is your most clever way to report the end of a business process? Tell us in the comments!

How to use LDAP in a Javalin Server

I recently implemented authentication and authorization via LDAP in my Javalin web server. I encountered a few pitfalls in the process. That is why I am sharing my experiences in this blog article.

Javalin

I used pac4j for the implementation. This is a modular library that allows you to replicate your own use case with different authenticators, clients and web server connection libraries. In this case I use “org.pac4j:pac4j-ldap” as authenticator, “org.pac4j:pac4j-http” as client and “org.pac4j:javalin-pac4j” as web server.

In combination with Javalin, pac4j independently manages the session and forwards it for authentication if you try to access a protected path.

var config = new LdapConfigFactory().build();
var callback = new CallbackHandler(config, null, true);

Javalin.create()
   .before("administration", new SecurityHandler(config, "FormClient", "admin"))
   .get("administration", ctx -> webappHandler.serveWebapp(ctx))
   .get("login", ctx -> webappHandler.serveWebapp(ctx))
   .get("forbidden", ctx -> webappHandler.serveWebapp(ctx))
   .get("callback", callback)
   .post("callback", callback)
   .start(7070);

In this example code the path to the administration is protected by the SecurityHandler. The “FormClient” indicates that in the event of missing authentication, the user is forwarded to a form for authentication. The specification “admin” defines that the user must also be authorized to the role “admin”.

LDAP Config Factory

I configured LDAP using my own ConfigFactory. Here, for example, I define the callback and login route. In addition, my self-written authorizer and http action adapter are assigned. I will go into these two areas in more detail below. The login form requires the authenticator here. For us, this is an LdapProfileService.

public class LdapConfigFactory implements ConfigFactory {
    @Override
    public Config build(Object... parameters) {
        var formClient = new FormClient("http://localhost:7070/login", createLdapProfileService());
        var clients = new Clients("http://localhost:7070/callback", formClient);
        var config = new Config(clients);

        config.setWebContextFactory(JEEContextFactory.INSTANCE);
        config.setSessionStoreFactory(JEESessionStoreFactory.INSTANCE);
        config.setProfileManagerFactory(ProfileManagerFactory.DEFAULT);
        config.addAuthorizer("admin", new LdapAuthorizer());
        config.setHttpActionAdapter(new HttpActionAdapter());

        return config;
    }
}

LDAP Profile Service

I implement a separate method for configure the service. The LDAP connection requires the url and a user for the connection and the query of the active directory. The LDAP connection is defined in the ConnectionConfig. It is also possible to activate TLS here, but in our case we use LDAPS.

The Distinguished Name must also be defined. Queries only search for users under this path.

private static LdapProfileService createLdapProfileService() {
    var url = "ldaps://test-ad.com";
    var baseDN = "OU=TEST,DC=schneide,DC=com";
    var user = "username";
    var password = "password";

    ConnectionConfig connConfig = ConnectionConfig.builder()
            .url(url)
            .connectionInitializers(new BindConnectionInitializer(user, new Credential(password)))
            .build();

    var connectionFactory = new DefaultConnectionFactory(connConfig);

    SearchDnResolver dnResolver = SearchDnResolver.builder()
            .factory(connectionFactory)
            .dn(baseDN)
            .filter("(displayName={user})")
            .subtreeSearch(true)
            .build();

    SimpleBindAuthenticationHandler authHandler = new SimpleBindAuthenticationHandler(connectionFactory);

    Authenticator authenticator = new Authenticator(dnResolver, authHandler);

    return new LdapProfileService(connectionFactory, authenticator, "memberOf,displayName,sAMAccountName", baseDN);
}

The SearchDNResolver is used to search for the user to be authenticated. A filter can be defined for the match with the user name. And, very importantly, the subtreeSearch must be activated. By default, it is set to false, which means that only users who appear exactly in the BaseDN are found.

The SimpleBindAuthenticationHandler can be used together with the Authenticator for authentication with user and password.

Finally, in the LdapProfileService, a comma-separated string can be used to define which attributes of a user should be queried after authentication and transferred to the user profile.

With all of these settings, you will be redirected to the login page when you try to accessing administration. The credentials is then matched against the active directory via LDAP and the user is authenticated. In addition, I want to check that the user is in the administrator group and therefore authorized. Unfortunately, pac4j cannot do this on its own because it cannot interpret the attributes as roles. That’s why I build my own authorizer.

Authorizer

public class LdapAuthorizer extends ProfileAuthorizer {
    @Override
    protected boolean isProfileAuthorized(WebContext context, SessionStore sessionStore, UserProfile profile) {
        var group = "CN=ADMIN_GROUP,OU=Groups,OU=TEST,DC=schneide,DC=com";
        var attribute = (List) profile.getAttribute("memberOf");
        return attribute.contains(group);
    }

    @Override
    public boolean isAuthorized(WebContext context, SessionStore sessionStore, List<UserProfile> profiles) {
        return isAnyAuthorized(context, sessionStore, profiles);
    }
}

The attributes defined in LdapProfileService can be found in the user profile. For authorization, I query the group memberships to check if the user is in the group. If the user has been successfully authorized, he is redirected to the administration page. Otherwise the http status code forbidden is returned.

Javalin Http Action Adapter

Since I want to display a separate page that shows the user the Forbidden, I build my own JavalinHttpActionAdapter.

public class HttpActionAdapter extends JavalinHttpActionAdapter {
    @Override
    public Void adapt(HttpAction action, WebContext webContext) {
        JavalinWebContext context = (JavalinWebContext) webContext;
        if(action.getCode() == HttpConstants.FORBIDDEN){
            context.getJavalinCtx().redirect("/forbidden");
            throw new RedirectResponse();
        }
        return super.adapt(action, context);
    }
}

This redirects the request to the Forbidden page instead of returning the status code.

Conclusion

Overall, the use of pac4j for authentication and authorization on javalin facilitates the work and works well. Unfortunately, the documentation is rather poor, especially for the LDAP module. So the setup was a bit of a journey of discovery and I had to spend a lot of time looking for the root cause of some problems like subtreeSearch.

Why we can’t dispose of every Manager, even in Tech fields

The world is in a place in which we gradually learn to question old patterns of societal behaviour and to look deeply into the meaning of what it is that we actually do. Especially in the technical / digital field, we see that our hands are not tied – most of the things we do, we can decide to do for an actual reason, or to find a way of disposing of exactly that – in what speed or level of disruption we consider appropriate.

While this holds for any level of detail, it is especially fruitful in these patterns where we do something for historical reason; the patterns that emerged as seeming to hold true for generation after generation – why should the present days be any different?

Now, I do not hate any traditions just for the reason of them being traditions. That stance is not useful, as a significant number of traditions have some underlying reasons why they work. Sometimes these are not obvious at all, and worse, sometimes everyone around us thinks they can tell use these reasons, while what they tell is are mostly empty phrases, placeholders, tranquilizers, red herrings, tautologies. But I digress.

One of the largest volume of “what we do” revolves around our work. Not only in a “this is a German clichee” sense, but the universal “what we deliver to sustain our livelihood”. And one of the most profitable discussions are these at the intersections of that work and of our inner state of being, our emotions, attitudes, values, interpersonal relations. The stuff that plays no role in traditional work.

To clarfiy, the world has not yet gone to a magical sphere of enlightenment where we’re just so incredibly smarter than our ancestors, but at least the reasoning takes place, and is within reach to be emplaced.

But there is little insight in only asking “what have they thought wrong?”. You have to also ask “what have they gotten right?” to get to the next level.

The Question: “What is Management?”

It’s not as simple to tell, because if you just look at what most Managers do, you can only answer “What is Mismanagement?”. Think of any fictional CEO, and chances are you will imagine one of these movie stereotypes, the people who put more work in buttering their insecure ego with layers of self-importance and complacency until the whole package somewhat sticks.

As such, there appears to be a current trend in Management Consultants that actually do God’s work in trying to correct that view. Do any internet search, there is plenty of quotable verdict on that field – “As a manager, you should trust your people and get out of their way. They work with you for a reason” – Even if they might not convince many Executives in changing their ways, more lower-employed people lose their tolerance for such mannerisms.

This means, that in a job interview, a Newcomer can very well ask – “Tell me, what do we need you for? Under the assumption that I’m the right person for this job, what is there I could possibly need an Executive for without invalidating the former assumption?” (… is not something your average applicant would say, but it’s the core of that idea that matters.)

So, there’s the obvious bulk of organisational overhead, outside representation, legal matters, ensuring-everyone-gets-paid-on-time, duties about Datenschutz & friends, i.e. tasks that are most efficiently handled by as few hands as possible. (Depending on how American you feel, there you might also need to constantly insult your competitors publicly.) But other than that, do you just hang around and ask your employees about their favorite variety of coffee once in a while?

  • Yes, in some instances that is the most preferable conduct for your Manager,
  • Generally, it’s not.

Where you can really reap the crops

Basically, you are deployed to optimize every flow of any relevant currency. These might change over time, but it is not only the regular appearance of some money in some accounts and it’s also not only the getting out of your way to let the recipients of said money do their magic. It’s also not about everyone feeling absolutely smart and awesome on every occasion. Neither the opposite.

You can be of actual use. Here’s three points that come to my mind.

  • 1.) Define the Culture, especially its Boundaries
  • 2.) Evaluate the Direction of Motion
  • 3.) Advertise that Words can have an Actual Meaning

Now I guess I have written a lot already, so I will have to delve into details in some upcoming posts. But let me summarize.

@1.) Defining the culture of your work place is something you either do conciously or fail miserably, but it gets exponentially more important over time.

  • Some obvious things you might not need to define (e.g.” no physical assault”)
  • Then there’s potential for micromanagent or personal overreach. These you should never interfere with or only decide case-by-case.
  • But some decisions in between have many options, still someone needs to break the symmetry. E.g. “use this particular tool for XYZ”. This can change, and you should always be able to give actual reasons, but if you don’t limit this choice, your people will waste their focus on the wrong questions, and one can only handle a certain number of question marks in their day. It’s about focus.

@2.) Evaluating the direction of motion of your business also sounds obvious. It more or less is. But there is not only motion in the sense of business output, e.g. “we focus on software that…” but inner direction, e.g. “we aim to recruit people who…”. These will change by factors outside your control and if you choose not to change course, you should do it on purpose, not by not thinking about it.

@3.) Knowing what you do (see 2) and how you do it (see 1) is quite nice already. Some focus is even nicer, but even more important than that is guaranteeing a level of consistency that guarantees that the foresaid boundaries are not subject to momentary reassessment. Do not think that one can ever act on a purely ration basis – this does not work ever – but treat your word very deliberate. This does not mean every actual vocabulary, but their fundamental underpinning. Not every attitude can hold forever either. Neither can you be perfect and never forget a thing. But if you appear careless in your word, people will notice that you consider them merely pawns in a game, and they might see that you don’t actually know how to play.

Now this became longer than intended. Anyway, I wish you a happy <legal reason why there’s no work next Monday>!

PS: this blog post is not sponsored or otherwise favorably promoted by the Softwareschneiderei management level 😉

Simple marching squares in C++

Marching squares is an algorithm to find the contour of a scalar field. For example, that can be a height-map and the resulting contour would be lines of a specific height known as ‘isolines’.

At the core of the algorithm is a lookup table that says which line segments to generate for a specific ’tile’ configuration. To make sense of that, you start with a convention on how your tile configuration and the resulting lines are encoded. I typically add a small piece of ASCII-art to explain that:

// c3-e3-c2
// |      |
// e0    e2
// |      |
// c0-e1-c1
//
// c are corner bits, e the edge indices

The input of our lookup table is a bitmask of which of the corners c are ‘in’ or above our isolevel. The output is which tile edges e to connect with line segments. That is either 0, 1 or 2 line segments, so we need to encode that many pairs. You could easily pack that into a 32-bit, but I am using a std::vector<std::uint8_t> for simplicity. Here’s the whole thing:

using config = std::vector<std::uint8_t>;
using config_lookup = std::array<config, 16>;
const config_lookup LOOKUP{
  config{},
  { 0, 1 },
  { 1, 2 },
  { 0, 2 },
  { 2, 3 },
  { 0, 1, 2, 3 },
  { 1, 3 },
  { 0, 3 },
  { 3, 0 },
  { 3, 1 },
  { 1, 2, 3, 0 },
  { 3, 2 },
  { 2, 0 },
  { 2, 1 },
  { 1, 0 },
  config{},
};

I usually want to generate index meshes, so I can easily connect edges later without comparing the floating-point coordinates. So one design goal here was to generate each point only once. Here is the top-level algorithm:

using point_id = std::tuple<int, int, bool>;

std::vector<v2<float>> points;
// Maps construction parameters to existing entries in points
std::unordered_map<point_id, std::uint16_t, key_hash> point_cache;
// Index pairs for the constructed edges
std::vector<std::uint16_t> edges;

auto [ex, ey] = map.size();
auto hx = ex-1;
auto hy = ey-1;

// Construct inner edges
for (int cy = 0; cy < hy; ++cy)
for (int cx = 0; cx < hx; ++cx)
{
  std::uint32_t key = 0;
  if (map(cx, cy) > threshold)
    key |= 1;
  if (map(cx + 1, cy) > threshold)
    key |= 2;
  if (map(cx + 1, cy + 1) > threshold)
    key |= 4;
  if (map(cx, cy + 1) > threshold)
    key |= 8;

  auto const& geometry = LOOKUP[key];

  for (auto each : geometry)
  {
    auto normalized_id = normalize_point(cx, cy, each);
    auto found = point_cache.find(normalized_id);
    if (found != point_cache.end())
    {
      edges.push_back(found->second);
    }
    else
    {
      auto index = static_cast<std::uint16_t>(points.size());
      points.push_back(build_point(map, threshold, normalized_id));
      edges.push_back(index);
      point_cache.insert({ normalized_id, index });
    }
  }
}

For each tile, we first figure out the lookup input-key by testing the 4 corners. We then get-or-create the global point for each edge point from the lookup.
Since each edge in a tile can be accessed from two sides, we first normalize it to have a unique key for our cache:

point_id normalize_point(int cx, int cy, std::uint8_t edge)
{
  switch (edge)
  {
  case 3:
    return { cx, cy + 1, false };
  case 2:
    return { cx + 1, cy, true };
  default:
    return { cx, cy, edge == 0 };
  };
}

When we need to create a point an edge, we interpolate to estimate where exactly the isoline intersects our tile-edge:

v2<float> build_point(raster_adaptor const& map, float threshold, point_id const& p)
{
  auto [x0, y0, vertical] = p;
  int x1 = x0, y1 = y0;
  if (vertical)
    y1++;
  else
    x1++;

  const auto s = map.scale();
  float h0 = map(x0, y0);
  float h1 = map(x1, y1);
  float lambda = (threshold - h0) / (h1 - h0);

  auto result = v2{ x0 * s, y0 * s };
  auto shift = lambda * s;
  if (vertical)
    result[1] += shift;
  else
    result[0] += shift;
  return result;
}

For a height-map, that’s about as good as you can get.

You can, however, sample other scalar field functions with this as well, for example sums of distances. This is not the most sophisticated implementation of marching squares, but it is reasonably simple and can easily be adapted to your needs.