Porting an application from Oracle to PostgreSQL

Currently, we are in the process of porting a larger application from an Oracle database to a PostgreSQL database as an additional database system. The aim is for the application to be functional not only with one database system but with both: with Oracle for one customer and with PostgreSQL for another customer.

The first step in this process was to conduct a high-level analysis of what effort this would entail and where the application would need to be adjusted. The application consists of multiple services. A frontend service in Ruby on Rails and a variety of backend services in Java. Rails uses an O/R mapper called Active Record, which is relatively database-agnostic. No major adaptation effort was expected here. Ultimately, only the Active Record adapter needs to be configured per instance.

The Java services do not use an O/R mapper. They make direct SQL queries to the target database system using JDBC. This is where the greatest potential for issues existed. Fortunately, all database queries here are encapsulated in repository classes, making the SQL queries easily accessible and adaptable for different database management systems. However, this has not proven necessary so far. The SQL queries are written in standard SQL, without using database-specific features. The code is limited to basic features such as tables, views, sequences, SELECTs, INSERTs, UPDATEs, and JOINS, all of which are supported by both databases. Particularly, no stored procedures are used, which wouldn’t necessarily be portable.

For implementation, the initial goal was to make the existing Oracle database schema of the application reproducible on a PostgreSQL database, in a way that a new instance can be set up fresh in a Docker container at any time.

The best way to achieve this is with a database migration tool. The options considered were: Active Record Migrations, Liquibase, and Flyway. We ultimately chose Liquibase, which we have had positive experiences with. It can be used standalone and allows for formulating migrations in a database-independent manner. Additionally, it offers the option to export an existing schema as a migrations changelog file using the generate-changelog command.

We only had to slightly adjust the generated changelog to make it work for both database systems. In a Liquibase changelog, you can define variables referred to as properties, which can take on different values for various database systems. One use case for this is having different names for column data types:

<property name="varchar" value="varchar" dbms="postgresql"/>
<property name="varchar" value="VARCHAR2" dbms="oracle"/>

These properties can then be used, for example, in the following way:

<column name="DESCRIPTION" type="${varchar}(4000)">

Here’s a helpful table that maps Oracle datatypes to their equivalent PostgreSQL data types. After some editing of the changelog, mainly through search and replace, we had a migration for the database schema that works for both systems.

Exploring Tango Admin Devices

We are using the open-source control system framework TANGO in several projects where coordinated control of multiple hardware systems is needed.

What is TANGO good for?

TANGO provides uniform, distributed access and control of heterogeneous hardware devices. It is object-oriented in nature and usually one hardware device is represented by one (or more) software devices.

The device drivers can be written in either C++, Java or Python and client libraries exist for all of these languages. Using a middleware adapter like TangoGQL any language can access the devices.

All that makes TANGO useful for building SCADA systems ranging from a handful controlled devices to several hundreds you want to supervise and control.

Lesser know features of TANGO

All of the above is well known in the TANGO and SCADA community and quite straightforward. What some people may not know is that TANGO automatically provides an Admin-device for each TANGO server (an executable running one or more TANGO devices).

These admin devices have an address of the form dserver/<server_name>/<instance_name> and provide numerous commands for controlling and querying the TANGO device server instance:

You can for example introspect the device server to find available device classes, device instances and needed device properties (think of them as configuration settings).

In addition to introspection you can also control some aspects of the TANGO server like polling and logging. The Admin-device also allows restarting individual devices or even the whole server instance. This can be very useful to apply configuration changes remotely without shell access or something similar to the remote machine.

Wrapping it up

Admin-devices automatically exist and run for each TANGO device server. Using them allows clients to explore what devices are available, what they offer and how they can be configured. They also allow some aspects to be changed remotely at runtime.

We use these features to provide a rich web-base UI for managing the control system in a convenient way instead of relying on the basic tools (like Jive and Astor) that TANGO offers out-of-the-box.

The Fragmented Sources of Truth for a Software Project

For each software application, there only exists one single, authoritative source of truth: The source code. If something isn’t in the code, it doesn’t exist. This source of truth is so important that we invented version control (or source control) that allows us to:

  • travel backwards in time
  • create alternative realities
  • progress multiple realities concurrently

That’s pretty awesome and something that not many professions can rely on. It is a “hidden superpower” of software development.

But when you look at a software project and not just the application, there is a lot more “truths” or information available than what fits into the source code. Let’s have a look at a few of them:

The ticket system

The ticket system or issue tracker or bug tracker or whatever you call it is a glorified to-do list that tells people what is lacking in the source code.

One view on the ticket system could be that of a health record system. Each ticket represents an ailment that the software application has. If it’s a bug, it is clearly in an undesirable state that needs to be “healed”. If it’s a new feature, the medical metaphor doesn’t fit perfectly, but we can view our development work as some kind of plastic surgery that makes the software more appealing to the customer.

Either way, the ticket system holds episodical wisdom. It explains the state of our application in hundreds or thousands of more or less independent short stories that are worked on in isolation. To gain a complete vision about the software project from the ticket system alone is possible, but cumbersome.

If you think about it, there is a clear connection between a short story (ticket) and one alternative reality in which the application is told about the story.

The wiki

For each project, there is a lot of information that is fluid, but not episodical. The current state of truth is valid until it gets replaced by a newer truth. Attempting to capture this information in tickets would result in an awkward lack of oversight.

Luckily, there is a tool that was invented specifically for this type of information: The wiki or the editable website graph. Your project can claim an area in this graph and fragment the information according to the mental model of the project team. Every time some outdated information is found, it can be updated in place. Every time some information is not found, it can be added in the place where it was anticipated.

The file storage

Every software project that I know has a lot of accompanying documents that are important for the project, but maybe not so much for the actual day-to-day development. Depending on who you ask, these documents may very well constitute “the project” and everything “below” them are just necessary technicalities.

The nature of a document is that it exists forever once created. There are lots of attempts to bring version control to the document world, but a typical question in this area is: “Is my document still valid?”

Because documents are represented by files (in the digital and the analog world), a file storage is the least we need to manage them. If your file storage entices you to name your documents “_latest”, “_newer”, “_version2” or something like that, you probably want to step up your document versioning game. Document management systems (DMS) might be what you are looking for. For small teams, a central instance of a Nextcloud might already be sufficient.

The derived documentation

If you happen to develop a software product, you need to provide a user manual and additional technical documentation. These documents need to be in eventual synchronisation with your first and central source of truth: The source code. And because your source code adapts, these documents need to adapt constantly, too.

This is the area where our company has the most “room for improvement”. I’m not diving into details here because I know our approaches are not sustainable.

Single source of truth?

The problem with this fragmented approach to capture the whole of a project is that you need to study all the different places and combine the information in your head. And not only you, every team member has to do this.

You can try to combine different sources into one:

If you squint really hard, you might think that a wiki can replace a ticket system, because each ticket can be represented by a graph node and the linking might resemble a grouping mechanism. My uninformed guess would be that this replaces software specialization with the need for human discipline. But maybe it can work and I just don’t know about the proper tooling yet?

One rather obvious integration might be to put the wiki alongside the code. I haven’t seen a good solution for merge conflicts yet, but maybe it is possible somehow.

Putting the file storage into your source repository makes it bigger and unwieldy, but it would be a natural step towards single sourcing – until you want to give your code repository away without revealing your company’s contracts. Suddenly, separate storage areas become important.

The one thing I struggle to integrate into the source repository is the derived documentation. I can think about storing the documents alongside the code and even requiring to update them before a merge request of a feature branch is accepted, but I shudder to think about the inevitable merge requests that need to be resolved.

Maybe there is a suitable solution out there that I’m missing? Leave a hint in the comments!

Oracle database identity column

A few days ago, I came across the topic of identity columns in Oracle. So I decided to try it out to check if it would benefit my database tables.  I would like to share the results in this article. First, I will explain what identity columns are and how to create them.

General

If you declare a column as an identity column, the value will be automatically populated from an associated sequence generator. It is possible to add an identity column by using either the CREATE TABLE or ALTER TABLE statement. In the example below, I will show how to add it using the CREATE TABLE statement.

The column marked as an identity needs to be of the data type integer, long, or number.

CREATE TABLE EXAMPLE (
    id NUMBER GENERATED ALWAYS AS IDENTITY,
    ...
);

In this case, I ALWAYS want a value to be automatically filled in. There is also the option to only fill it if no value was specified (BY DEFAULT) or when the given value is null (BY DEFAULT ON NULL).

Additionally, you can configure the sequence as shown in the following example.

CREATE TABLE EXAMPLE (
    id NUMBER GENERATED ALWAYS AS IDENTITY
    (START WITH 1 INCREMENT BY 2 MAXVALUE 5000 NO CYCLE CACHE 100),
    ...
);

With these  settings, the sequence starts with 1 and increments by 2, so the values are 1, 3, 5, and so on. The sequence counts up to 2000 and then stop. If CYCLE is set, the sequence continues with 1 again.

The cache defines a client to request 100 pre-defined values in advance. Because of the step width of 2 for example the values 1 – 199 will be assigned. If a second client asks, it will receive the values 201 – 399. As a result, the entries in the table can have the IDs 1, 201, 3, 5, 7, 203 for example.

Using with Ruby on Rails / Active Records

My use case is to use the table in Ruby on Rails with Active Records. Here, I want Active Record to auto-increment the ID, so I can retrieve it in my program. However, if I do not use the identity column feature and also do not specify the name of a manually created sequence, Ruby always returns 0 as the ID. When I add a sequence name, I get the correct ID.

Now I am trying to change this to work with the identity column. The insert in Oracle directly with SQL works fine.

The next step is to use it in my Ruby program, and the first problem occurs. Unfortunately, it is not possible to define my own sequence name. Oracle automatically creates sequences with cryptic names like ISEQ$$_12345 for identity columns which reduces readability in code. Additionally if I recreate the table, the sequence name changes, which means I have to update my code afterwards. This seems to be the first showstopper.

Nonetheless I continue and try to create a new entry with it. The entry was created. It works! Or wait a moment. Ruby always shows 0 instead of the right ID. It seems like the program can not use the sequence correctly.  I halt my testing. Maybe it is possible to make Ruby work with identity sequences, maybe it is not. If you have already found a solution, feel free to comment.

In the end I decide that using the manually created sequence still fit better. The main reason is the naming, so I do not want to spend more time searching for a solution to make it work with Ruby.

Conclusion

Although the identity column option does not fit my use case, it is a possible to create a table column with auto-generated values in Oracle. For situations where the ID is not of interest, it can simplify the work. You do not have to create the logic with sequences and triggers on your own.

Interacting with SVG files inside your React applications

So, for some reason you have a SVG file that somehow resembles a part of the application you are currently developing.

This might be only a sketch that you want to prepare as a Click Dummy, or it might be that you need to display a somewhat unique, complicated structure that is best layed out per SVG editor. This is somewhat expected when you have customers in the technical / scientific research sector.

So now you want to fill it with life.

Now, the SVG format is quite close to the <svg> structure that one can embed into HTML, but there are some steps in between. Most importantly, most SVG Editors fill their .svg files up with meta data or specific information only required for the editor in case you want to edit the files again.

Thus, you have three choices to integrate the SVG component in a React application

  • Re-Build your SVG with custom React Components that, via JSX, render their <svg>, <g>, <path>, etc. accordingly
  • Convert your SVG to valid JSX – this is possible in many cases, but you need to take care e.g. that the style attribute is a string in the SVG and an object in JSX, also it can be still way too large to be readily put in a single React Component
  • Import your .svg as its own React Component and then wrap that into an own Component that takes care about the interaction part

While I also have written a small converter that does the SVG-JSX-Conversion just fine for me for any file that comes out of Inkscape (probably an idea for my next blog post), we had the case of some files with about 16000 lines of SVG each, so I chose the third method in our case.

In my eyes, it is very correct to mention that the following way somehow goes against the React Mindset. In which you render all your components yourself to attach them the required mouse event handlers, never having to interact with a HTML “id” or any document.getElementById() or document.getElementByClassName().

In any React application, these should be avoided, but the idea here is to have a singular point – a SvgWrapper Component – where you allow these functions because you’d agree about the need to somehow target the specific SVG elements.

The gist:

import {ReactComponent as OurHorrificSvgMonster} from "/src/monster.svg";

const OurBeautifulComponent = () => {

    useOurCarefulSvgSynchronizationEffect(); // more on this below

    return (
        <SvgWrapper>
             <OurHorrificSvgMonster/>
        </SvgWrapper>
    )
};

Quick note: you can target the embedded <svg> element itself by <OurHorrificSvgMonster ref={...}/> and you could use this (ref.current holds that HTML element) to traverse all the children, so if you know much about the structure of your svg you could even live without the <SvgWrapper>. But say someone else made the horrific svg monster and all you have is the id or class names to all the individual svg elements inside.

Then

const WithVanillaHandlersConnected = ({children}) => {
    const dispatch = useDispatch();

    React.useEffect(() => {

        const onClick = (event) => {
            dispatch(awesomeAction(event.target.id));
        };

        const awesomeElements = [...document.getElementsByClassName("awesome")];

        awesomeElements.forEach(elem => {
            elem.addEventListener("click", onClick);
        });

        return () => awesomeElements.forEach(pipe => {
            pipe.removeEventListener("click", onClick);
        });
    }, []);

    return children;
};

I chose this dispatch() as a placeholder for any interaction with the surrounding web application, it could also be a simple React state or something. You can register any event listener you want here (also “mouseover”, “mouseout”, “contextmenu”, …), but think of removing it again in the effect return function.

By the way, document.getElementsByClassName(…) returns something like a HTMLCollection which is not exactly iterable, thus the […destructuring] to make the .forEach() possible.

We now have the first part – our elements (in our case, everything that has class “awesome”) has got a click handler that allows to dispatch anything to the application state. But now they need to change, too.

In a purely React-y way, this could be done by a svg element that chooses its fill = {isActive? "magenta" : "black"} but as we choose not to render our components ourselves, we need to once again grab deeply into the DOM and dare to manipulate it by hand.

As mentioned above – this is a step towards very ugly problems as React cannot guarantee that your visual layer matches your application state. You, on yoru own, have to guarantuee to do what’s right.

This is where this comes in:

/*
 for this example, think of that the redux selector selectSomethingFromTheState returns something like:

result = [
  {elementId: "elem1", isActive: true},
  ...
];

and isActive could be the thing that was toggled by our awesomeAction() above

*/

const useOurCarefulSvgSynchronizationEffect = () => {
    const elementStates = useSelector(selectSomethingFromTheState);

    React.useEffect(() => {
        for (const state of elementStates) {
            const element = document.getElementById(state.elementId};
            element.style.fill = isActive ? "magenta" : "black";
            // ... do other stuff with the DOM element
        }
    }, [elementStates]);
};

There we have it – we have the SvgWrapper and the use…SynchronizationEffect() that both stray from the React mindset by accessing the DOM directly, but we do it in a fashion where it is concisely encapsulated inside <OurBeautifulComponent> and there is no direct knowledge about the IDs inside the SVG, or Class manipulations, CSS Selectors, etc. elsewhere.

In my opinion, one can indeed go against the rules if it’s necessary, but I also see the option for a Stockton Rush quotation here.. so, if you know of any more elegant way, please feel free to share.

PS: by the way, if you use vite, you might get an “Uncaught SyntaxError” when trying import { ReactComponent ... } – I’ve written about this before.

Naming is hard and Java Enums don’t help

This is a short blog post about a bug in my code that stumped me for some moments. I try to tell it in a manner where you can follow the story and try to find the solution before I reveal it. You can also just read along and learn something about Java Enums and my coding style.

A code structure that I use sometimes is the Enum type that implements an interface:

public enum BuiltinTopic implements Topic {

    administration("Administration"),
    userStatistics("User Statistics"),
    ;
	
    private final String denotation;

    private BuiltinTopic(String denotation) {
        this.denotation = denotation;
    }
	
    @Override
    public String denotation() {
        return this.denotation;
    }
}

The Topic interface is nothing special in this example. It serves as a decoupling layer for the (often large) part of client code that doesn’t need to know about any specifics that stem from the Enum type. It helps with writing tests that aren’t coupled to locked-down types like Enums. It is just some lines of code:

public interface Topic {

    String denotation();
}

Right now, everything is fine. The problems start when I discovered that the denotation text is suited for the user interface, but not for the configuration. In order to be used in the configuration section of the application, it must not contain spaces. Ok, so let’s introduce a name concept and derive it from the denotation:

public interface Topic {

    String denotation();
	
    default String name() {
        return Without.spaces(denotation());
    }
}

I’ve chosen a default method in the interface so that all subclasses have the same behaviour. The Without.spaces() method does exactly what the name implies.

The new method works well in tests:

@Test
public void name_contains_no_spaces() {
    Topic target = () -> "User Statistics";
    assertEquals(
       "UserStatistics",
       target.name()
    );
}

The perplexing thing was that it didn’t work in production. The names that were used to look up the configuration entries didn’t match the expected ones. The capitalization was wrong!

To illustrate the effect, take a look at the following test:

@Test
public void name_contains_no_spaces() {
    Topic target = BuiltinTopic.userStatistics;
    assertEquals(
        "userStatistics",
        target.name()
    );
}

You can probably spot the difference in the assertion. It is “userStatistics” instead of “UserStatistics”. For a computer, that’s a whole different text. Why does the capitalization of the name change from testing to production?

The answer lies in the initialization of the test’s target variable:

In the first test, I use an ad-hoc subtype of Topic.

In the second test and in production, I use an object of type BuiltinTopic. This object is an instance of an Enum.

In Java, Enum classes and Enum objects are enriched with automatically generated methods. One of these methods equip Enum instances with a name() method that has a default implementation to return the Enum instances’ variable/constant name. Which in my case is “userStatistics”, the same string I expect, minus the correct capitalization of the first character.

If I had named the Enum instance “UserStatistics”, everything would have worked out until somebody changes the name or adds another instance with a slight difference in naming.

If I had named my Enum instance something totally different like “topic2”, it would have been obvious. But in this case, with only the minor deviation, I was compelled to search for problems elsewhere.

The problem is that the auto-generated name() method overwrites my default method, but only in cases of real Enum instances.

So I thought hard about the name of the name() method and decided that I don’t really want a name(), I want an identifier(). And that made the problem go away:

public interface Topic {

    String denotation();
	
    default String identifier() {
        return Without.spaces(denotation());
    }
}

Because the configuration code only refers to the Topic type, it cannot see the name() method anymore and only uses the identifier() that creates the correct strings.

I don’t see any (sane) way to prohibit the Java Enum from automatically overwriting my methods when the signature matches. So it feels natural to sidestep the problem by changing names.

Which shows once more that naming is hard. And soft-restricting certain names like Java Enums do doesn’t lighten the burden for the programmer.

Table inheritance in PostgreSQL

In a previous post I have written about tables as data types in PostgreSQL. In addition to that, PostgreSQL has a feature similar to the inheritance mechanism of data types in object-oriented programming: table inheritance.

Table inheritance allows you to create a hierarchical relationship between tables. With this feature you define a parent table, and child tables inherit columns and some constraints (like CHECK constraints and NOT NULL constraints) from it.

How it works

To begin, we create the parent table products using the following SQL code:

CREATE TABLE products (
  id     SERIAL         PRIMARY KEY,
  name   TEXT           NOT NULL,
  price  DECIMAL(10, 2) NOT NULL
);

Next, let’s create child tables that inherit from the products table. We will create two product categories: electronics and clothing. Each child table will have its own specific columns in addition to the inherited columns. To set up the inheritance relation we use the INHERITS keyword:

CREATE TABLE electronics (
  warranty_period   INT,
  power_consumption INT
) INHERITS (products);

CREATE TABLE clothing (
  size  TEXT,
  color TEXT
) INHERITS (products);

Now we insert some rows into the child tables:

INSERT INTO electronics
  (name, price, warranty_period, power_consumption)
  VALUES ('Laptop', 1439.0, 2, 536);

INSERT INTO clothing
  (name, price, size, color)
  VALUES ('T-Shirt', 5.99, 'L', 'red');

When we query each child table individually, we only get the rows we inserted into it, as expected. However, when we query the parent table, we get all the products from the child the tables:

SELECT * FROM products;

id	name	 price
--------------------
1	Laptop	 1439.00
2	T-Shirt	 5.99

Note that you can also insert rows into the parent table:

INSERT INTO products
  (name, price)
  VALUES ('Milk', 1.15);

This row will not show up in the child tables, only when querying the parent table. If you are asking yourself now if there is an equivalent to an “abstract class”, there isn’t one. Just don’t insert into the parent table if it doesn’t make sense in your domain.

If you want to query rows only from the parent table you can use the ONLY keyword:

SELECT name, price FROM ONLY products;

This keyword works also with other commands like UPDATE and DELETE.

On a final note, you can also remove the inheritance relationship from a child table with the NO INHERIT clause:

ALTER TABLE clothing NO INHERIT products;

Trying ChatGPT as a professional software developer

We are a company of software developers that does not sell a product but provides a wide range of services. Our primary service is developing individual software or maintaining existing software for our customers.

Most of us are polyglot developers each sufficiently fluent in a couple of languages. While I see this as a strength it also means that I do not know the idiomatic solution for a task in all ecosystems in use in our projects by heart.

A few days ago I had a remote-pair-programming session with one of our customers and we wondered how to idiomatically trim certain characters from a string in JavaScript. My customer came up with the idea of asking ChatGPT for a solution while I would have asked Google or stackoverflow.

Being curious we took the ChatGPT route and it delivered a correct solution after I clarified what I mean with “trim”:

function trimCharacterFromStartAndEnd(str, char) {
  // Create a regular expression to match the specified character at the beginning and end of the string
  const regex = new RegExp(`^${char}+|${char}+$`, 'g');
  
  // Use the replace() function to remove the character from the start and end of the string
  const trimmedStr = str.replace(regex, '');
  
  return trimmedStr;
}

// Example usage
const originalString = '###Hello World!###';
const trimmedString = trimCharacterFromStartAndEnd(originalString, '#');
console.log(trimmedString); // Output: Hello World!

But from a professional perspective the code was not simple, concise and idiomatic. I intuitively knew there must be a better way.

So I did what ChatGPT does best: I chatted with it to improve the solution to arrive at a solution a was satisfied with:

function trimCharacterFromStartAndEnd(str, char) {
  return str.replace(/^${char}+|${char}+$/g, '');
}

// Example usage
const originalString = '###Hello World!###';
const trimmedString = trimCharacterFromStartAndEnd(originalString, '#');
console.log(trimmedString); // Output: Hello World!

However, you possibly need to handle regex special characters like '.', '*' etc. if they can part of your characters to trim.

Some of the intermediate steps also have their uses depending on the needed flexibility. See the full conversation at trim character from string chat.

Similarily, stackoverflow provides some comprehensive answers you can adapt to your specific situation.

Evaluation

Using ChatGPT can actually provide you useful results. To make the most out of it, you have to be able to judge the solution provided by the AI and try to push it in the wanted direction.

After my experiment our students got the inofficial advice that their solutions should not be worse than what ChatGPT delivers. 😀

Arriving at a good solution was not faster or easier than the traditional developers’ approach using Google and/or stackoverflow. Nevertheless it was more interactive, more fun and most importantly it worked.

It was a bit disappointing to lose context at some points in the conversation, with the g-flag for example. Also the “shortest” solution is longer than the variant with the regex-literal, so strictly speaking ChatGPT’s answer is wrong…

I will not radically change my style of work and jump on the AI-hype-train but I plan to continue experimenting with it every now and then.

ChatGPT and friends certainly have some potential depending on the use case but still require a competent human to judge and check the results.

My biggest decision as a business owner (yet)

This week, a very fortunate event will take at our company: We all come together to have a summer party in person. This will be the first time in nearly 3 and a half years that we all spend time in the same room. It will be the conclusion of a decision that I call the “biggest one” that I had to come to. This is the very shortened story of that decision.

The end of an era

Our company was founded and set up as a place for direct interaction and short communication distances. We favored office workplaces and open space room plans and often visited customers at their location.

In March 2020, this setup appeared to be the exact opposite of what is advised. I remember the week from the 9th to the 13th March, when every day and every hour, things got worse and more restricted due to the Sars-Cov2 pandemic. On Friday, the 13th of March 2020, I was in a phone call with an employee that lasted 30 minutes. When we began to speak, one federal state had closed the schools. When we stopped, every school was closed in the whole country.

During the weekend, I tried to approach the situation with plans and lists. A list of endangered projects, a list of endangered customers, a list of endangered employees, a list of critical tasks, a plan to stay ahead of circumstances. I came up with a scheme to assess the risk and derive actions, but spent the whole sunday to talk with my employees just to gather some of the information necessary to base any decision on more than fear and hope. I am very grateful that my employees all picked up the phone and went through my questions with me. It helped me to realize that no matter how fitting the lists, how clever the plan, I won’t be able to process the information with the required speed.

Some employees offered to go on holiday to take moving parts out of the equation, but it was still overwhelming. If you know the feeling in a roller coaster when a certain “feel-good” speed limit is exceeded and real fear takes hold of your heart and head, you can imagine how these days felt for me.

The beginning of a different era

And then, on Monday morning, I knew exactly what to do. The situation necessitates that we change everything at the company at once. We need to go “virtual”, to retreat into home offices that didn’t exist yet.

Monday, 16th of March was the last day that several people were in our office simultaneously for a long time.

Everything the company was used to do didn’t work anymore. We had to buy new hardware, new furniture, new chairs and everything else that was needed in the home offices. We had to examine every business process and partition it into “on site” and “remote” work steps. We had to introduce new means of communication in the company and with our customers. We had to continue with our project work while transforming everything in our professional and our private lifes. We had to keep up our spirits while experiencing isolation and uncertainty.

And just like that, we replaced the “pre-covid” company with the “during-covid” company. Nobody could say that it would work. Nobody knew how long it would be required to work. Nobody could anticipate how much it would cost us.

The decision

The only thing I was certain of was that if we need to change, we would do it wholeheartedly. I was sure that even if the pandemic suddenly disappears, I don’t want to look back at that time and think of it as a makeshift solution.

My decision was to embrace the uncertainty and let go of any remnant of a masterplan that I might have left. I “jumped into the fog”.

For me, it felt as if I placed a wager on the existence of the whole company: “I bet we can do what we did for twenty years, but totally different and in a time of crisis. And we can start right now and keep going for an indeterminate period of time”.

The outcome

Since then, a long time has passed. The fog has cleared and we have survived. And not only that. The “gamble” has paid off:

We resumed our project work within two days and steadily improved our situation day by day and week by week. Our revenue went up, our productivity went up, our profits went up. New customers called us, new projects were started. Today, we are in a much better place than before.

But that’s not all: We have established new means of collaboration and communication, regardless of workplace. Every employee has a full-fledged home office with as many monitors as are physically possible, fitting furniture, a good webcam, good audio equipment, a powerful notebook or desktop computer and all the accessories that make the difference between “a workplace” and “my own workplace”. So we are fully equipped for any future isolation event that hopefully never comes.

Making the decision, trusting my employees and providing them with the equipment to master the challenge yielded the best outcome I could have hoped for. The whole experience humbled me: I lost any control over the situation early on and it didn’t really matter. What mattered was to keep innovating, investing and improving. And that is a group effort, not the vision of a single mind.

The future

So, here we are, at the natural end of the story. If this was a movie, the credits would begin to roll when we raise our glasses to celebrate our success. To me, it seems that lots of companies operate like this. “The temporarily embarrassing loss of control of upper management is past, now return to the office and commence the old rituals. And don’t forget to bring in that notebook that we borrowed you for your kitchen table home office.”

I’ve seen the potential of this transformation way too clearly to go back. There is nothing gained by reverting to the old ways. We will continue as a “hybrid” company with an attractive office and equally attractive home offices. We will continue to find ways to collaborate with each other and our customers that we didn’t think of before. We will continue to spend time, effort and money to improve our work reality. It might cost 15k euros to equip one workplace in the office and 15k euros more to do it again at home, but that money is the best investment I can think of. The return on investment is amazing.

I witnessed it firsthand.

JSON Web Token (JWT) and Security

General

JWT is an open standard for transmitting information as a JSON object. The most common scenario is authorization.

Unfortunately, the token keeps cropping up in connection with the security vulnerability. For example, it is mentioned in the OWASP top ten under the item “Broken Access Control”. In the following, I would like to briefly explain the JWT and point out a few risks when using it.

The token consists of three parts: the header, payload, and signature. Each part has been encoded with Base64Url, and all parts are joined together, separated by dots.

The type of the token and the signing algorithm is typically defined in the header.

{
  "alg": "HS256",
  "typ": "JWT"
}

The payload contains the information that should be transmitted. In cases of authorization, the user and permissions. It is also possible to define token metadata, like an expiration time. Such information must be checked by the developer. The token itself has no expiration.

{
  "exp": 1516242622,
  "name": "Max Mustermann",
  "admin": true
}

The signature take the encoded header, the encodes payload and a secret and encode it with the algorithm defined in header.

HMACSHA256(
  base64UrlEncode(header) + "." +
  base64UrlEncode(payload),
  secret)

In the authorization scenario, the token is requested from the client and created by the authorization server. The client uses the token to authorize a resource server. Therefore, the token is sent in the authorization header using the bearer schema.

Security Issues

JWT has a lot of attack surfaces, like the None Hashing Algorithm, ECDSA “Psychic Signatures”, Weak HMAC Keys, HMAC vs Public Key Confusion, Attacker Provided Public Key and the plaintext transmitting of the payload. All in all, JWT is vulnerable to replaying or tampering. Especially if the secret is not strong, the token does not expire, or no signature is used at all.

In the following, I will take a closer look at the none hashing algorithm issue as an example:

JWT allows you to set the algorithm in the header to none. So the signature part that tries to detect tampering is missing. It is just the first two parts and the point of waiting for a signature, which does not come.

eyJhbGciOiAibm9uZSIsICJ0eXAiOiAiSldUIn0K.eyJ1c2VybmFtZSI6ImFkbWluaW5pc3RyYXRvciIsImlzX2FkbWluIjp0cnVlLCJpYXQiOjE1MTYyMzkwMjIsImV4cCI6MTUxNjI0MjYyMn0.

All security precautions are already suspended, and the attacker can create his own tokens or modify intercepted ones at his leisure.

Some frameworks forbid this configuration, but in the JWT standard, it is possible. Although many applications and libraries that use JWT have disabled the option or at least tried to do so, there are still security vulnerabilities due to the configuration option. For example, because only case-sensitive “none”  was checked, settings like “None” or “NoNe” could still be used as an attack surface. So if you want to forbid this setting, it is important to do it case-insensitively.

Since this is a big problem that also occurs again and again, there is a website that counts the days since the last none algorithm problem: https://www.howmanydayssinceajwtalgnonevuln.com/.

The none algorithm is not the only attack surface on the algorithm usage. Most of the before named issues are related to the algorithm part.

How to use most secure

At best, the token should only be used once to confirm authentication and authorization. It should not be used for session management and has a short lifespan. Thus, the time span for attacks can be minimized. Also, the algorithm none option should be prevented, and a proper signature algorithm should be used. Additional information and configuration recommendations can be found under: https://jwt.io/ and OWASP cheatsheet.