Web Components, Part 2: Encapsulating and Reusing common Element Structure

In my previous post, I gave you some first impressions about custom HTML Web Components. I cut myself short there to make my actual point, but one can surely extend this experiment.

The thing about the DOM is that it is one large, global block of information. In order to achieve loose coupling, you need to exert that discipline by yourself, using document.getElementById() and friends you can easily couple the furthestmost components, to their inner workings, together. Which can make it very insecure to change.

For that problem in Web Components, there is the Shadow DOM. I.e. if you define, as previously, your component as

class CustomIcon extends HTMLElement {
    connectedCallback() {
        this.innerHTML = `
            <svg id="icon">
                <!-- some content -->
            </div>
        `;
        element = document.getElementById("icon");
        element.addEventListener(...);
        // don't forget to removeEventListener(...) in disconnectedCallback()! - but that is not the point here
    }
}

it becomes possible to also document.getElementById("icon") from anywhere globally. Especially with such generic identifiers, you really do not want to leak your inner workings. (Yes, in a very custom application, there might be valid cases of desired behaviour, but then usually the IDs are named as e.g. __framework_global_timeout, custom--modal-dialog, … as to avoid accidental clashes).

This is done as easy as

class CustomIconim d extends HTMLElement {
    constructor() {
        super();
        this.attachShadow({ mode: 'open' });
  }

    connectedCallback() {
        this.shadowRoot.innerHTML = ... // your HTML ere
    }
}

Two points:

  • The attachShadow() can also be called in the connectedCallback(), even if usually not required. Generally, there is some debate between these two options, and I think I’ll write you another episode of this post when I have some further insight about that.
  • The {mode: 'open'} is what you actually use because ‘closed’ does not give you that much benefit, as outlined in this blog here. Just keep in mind that yes, it’s still JavaScript – you can access the shadowRoot object from the outside and then still do your shenanigans, but at least you can’t claim to have done so by accident.

This encapsulation makes it easier to write reusable code, i.e. decrease duplication.

As with my case of the MagicSparkles icon – I might want to implement some other (e.g. Font Awesome) icons and have all of these carry the same “size” attribute. It might look like:

export const addSvgPathAsShadow = (element: HTMLElement, { children, viewBox, defaultColor }: SvgIconProps) => {
    const shadow = element.attachShadow({ mode: "open" });
    const size = element.getAttribute("size") || 24;
    const color = defaultColor || "currentcolor";
    viewBox ||= `0 0 ${size} ${size}`;
    shadow.innerHTML = `
            <svg
                xmlns="http://www.w3.org/2000/svg"
                width="${size}"
                height="${size}"
                viewBox="${viewBox}"
                fill="${color}"
            >
                ${children}
            </svg>
        `;
};

export class PlayIcon extends HTMLElement {
    connectedCallback() {
        addSvgPathAsShadow(this, {
            viewBox: "0 0 24 24",
            children: "<path fill-rule=\"evenodd\" d=\"M4.5 5.653c0-1.427 1.529-2.33 2.779-1.643l11.54 6.347c1.295.712 1.295 2.573 0 3.286L7.28 19.99c-1.25.687-2.779-.217-2.779-1.643V5.653Z\" clip-rule=\"evenodd\" />"
        });
    }
}

// other elements can be defined similarly

// don't forget to actually define the element tag somewhere top-level, as with:
// customElements.define("play-icon", PlayIcon);

Note that

  • this way, children is required as a fixed string. My experiment didn’t work out yet how to use the "<slot></slot>" here (to pass the children given by e.g. <play-icon>Play!</play-icon>)
  • Also, I specifically use the || operator for the default values – not the ?? – as an attribute given as empty string would not be defaulted otherwise (?? only checks for undefined or null).
Conclusion

As concluded in my first post, we see that one tends to recreate the same patterns as already known from the existing frameworks, or software architecture in general. The tools are there to increase encapsulation, decrease coupling, decrease duplication, but there’s still no real reason why not just to use one of the frameworks.

There might be at some point, when framework fatigue is too much to bear, but try to decide wisely.

Swagger-ui for any JVM-based backend

We often implement web applications with a React frontend and one of a large pool of backend frameworks/technologies. These include Micronaut, .NET Framework, Javalin, Flask, Eclipse Jetty among others.

A documentation of the API that allows calling the endpoints can be very helpful even during development to illustrate the API usage. OpenAPI and implementations like Swagger and Swashbuckle fulfill this task quite well.

While many of the backend frameworks support documenting and calling the API using Swagger-UI out-of-the-box or using plugins some frameworks like Undertow do not have direct support for it. Nevertheless, it is relatively easy to add an OpenAPI documentation with a web interface to almost any backend. We demonstrate the setup here using Undertow because it is used in some of our projects.

Adding dependencies and build config

Since we mostly use Gradle for our JVM-based backends we will highlight the additions to build.gradle to make to generate the OpenAPI definition. It consists of the following steps:

  • Adding the Swagger gradle plugin
  • Adding dependencies for the required annotations
  • Configuring the resolve-task of the gradle plugin

Here is an example exerpt of our build.gradle:

plugins {
  id 'java'
  id 'application'
// ...
  id "io.swagger.core.v3.swagger-gradle-plugin" version "2.2.20"
}

dependencies {
// ...
    implementation 'io.undertow:undertow-core:2.3.12.Final'
    implementation 'io.undertow:undertow-servlet:2.3.12.Final'
    implementation 'io.swagger.core.v3:swagger-annotations:2.2.20'
    implementation 'org.jboss.resteasy:jaxrs-api:3.0.12.Final'
}

resolve {
    outputFileName = 'demo-server'
    outputFormat = 'JSON'
    prettyPrint = 'TRUE'
    classpath = sourceSets.main.runtimeClasspath
    resourcePackages = ['com.schneide.demo.server', 'com.schneide.demo.server.api']
    outputDir = layout.buildDirectory.dir('resources/main/swagger').get().asFile
}

Adding the annotations

We need to add some annotations to our code so that the OpenAPI JSON (or YAML) file will be generated.

The API root class looks like below:

@OpenAPIDefinition(info =
@Info(
        title = "Demo Server Web-API",
        version = "0.11",
        description = "REST API for the demo web application..",
        contact = @Contact(
                url = "https://www.softwareschneiderei.de",
                name = "Softwareschneiderei GmbH",
                email = "kontakt@softwareschneiderei.de")
)
)
public class ApiHandler {
    public ApiHandler() {
        super();
    }

    /**
     *  Connect our handlers
     */
    public RoutingHandler createApiHandler() {
        final RoutingHandler api = new RoutingHandler();
        api.get("/demo", new DemoHandler());
        // ...
        return api;
    }
}

We also refactored our handlers to separate the business api and the Undertow handler interface methods to generate a expressive API.

The result looks something like this:

@Path("/api/demo")
public class DemoHandler implements HttpHandler {
    public DemoHandler() {
        super();
    }

    @Override
    public void handleRequest(HttpServerExchange exchange) throws Exception {
         exchange.getResponseHeaders().put(Headers.CONTENT_TYPE, "application/json");
            final Map<String, Deque<String>> params = exchange.getQueryParameters();
            final int month = Integer.parseInt(params.get("month").getFirst());
            final int year = Integer.parseInt(params.get("year").getFirst());
        exchange.getResponseSender().send(new Gson().toJson(getDaysIn(year, month)));
    }

    @GET
    @Operation(
            summary = "Get number of days in the given month and year",
            responses = {
                    @ApiResponse(
                            responseCode = "200",
                            description = "A number of days in the given month and year",
                            content = @Content(mediaType = "application/json",
                                    schema = @Schema(implementation = Integer.class)
                            )
                    )
            }
    )
    public Integer getDaysIn(@QueryParam("year") int year, @QueryParam("year") int month) {
        return YearMonth.of(year, month).lengthOfMonth();
    }
}

When running the resolve task all of the above results in a OpenAPI definition file in build/resources/main/swagger/demo-server.json.

Swagger-UI for the API definition

Now that we have this API definition we can use it to generate clients and – more important to us – generate a web UI documenting the API and allowing to execute and demo the functionality. For this we simply download the Swagger-UI distribution and place the contents of the dist/ folder in src/main/resources/swagger-ui. We then have to let Undertow serve definition and UI like so:

class DemoServer {
    public DemoServer() {
        final GracefulShutdownHandler rootHandler = gracefulShutdown(createHandler());
        Undertow.builder().addHttpListener(8080, "localhost").setHandler(rootHandler).build().start();
    }

    private HttpHandler createHandler() {
        return path()
                .addPrefixPath("/api", new ApiHandler().createApiHandler())
                .addPrefixPath("/swagger-ui", resource(new ClassPathResourceManager(getClass().getClassLoader(), "swagger-ui/"))
                        .setWelcomeFiles("index.html"))
                .addPrefixPath("/swagger", resource(new ClassPathResourceManager(getClass().getClassLoader(), "swagger/")));
    }
}

Note: I tried using the swagger-ui webjar but was unable to configure the location (the URL) of my OpenAPI definition file. Therefore I used the plain swagger-ui download instead.

Wrapping it up

We have to do some setup work and potentially some refactorings to provide a meaninful API documentation for our backend. After this work it is mostly adding some Annotations to methods and types used in your web API.

Web Components – Reusable HTML without any framework magic, Part 1

Lately, I decided to do the frontend for a very small web application while learning something new, and, for a while, tried doing everything without any framework at all.

This worked only for so long (not very), but along the way, I found some joy in figuring out sensible workflows without the well-worn standards that React, Svelte and the likes give you. See the last paragraph for a quick comment about some judgement.

Now many anything-web-dev-related people might have heard of Web Components, with their custom HTML elements that are mostly supported in the popular browsers.

Has anyone used them, though? I personally haven’t had, and now I did. My use case was pretty easy – I wanted several icons, and wanted to be able to style them in a unified fashion.

It shouldn’t be too ugly, so why not take something like Font Awesome or heroicons, these give you pure SVG elements but now I have the Font Awesmoe “Magic Sparkles Wand” like

<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 576 512"><!--!Font Awesome Free 6.5.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free Copyright 2024 Fonticons, Inc.--><path d="M234.7 42.7L197 56.8c-3 1.1-5 4-5 7.2s2 6.1 5 7.2l37.7 14.1L248.8 123c1.1 3 4 5 7.2 5s6.1-2 7.2-5l14.1-37.7L315 71.2c3-1.1 5-4 5-7.2s-2-6.1-5-7.2L277.3 42.7 263.2 5c-1.1-3-4-5-7.2-5s-6.1 2-7.2 5L234.7 42.7zM46.1 395.4c-18.7 18.7-18.7 49.1 0 67.9l34.6 34.6c18.7 18.7 49.1 18.7 67.9 0L529.9 116.5c18.7-18.7 18.7-49.1 0-67.9L495.3 14.1c-18.7-18.7-49.1-18.7-67.9 0L46.1 395.4zM484.6 82.6l-105 105-23.3-23.3 105-105 23.3 23.3zM7.5 117.2C3 118.9 0 123.2 0 128s3 9.1 7.5 10.8L64 160l21.2 56.5c1.7 4.5 6 7.5 10.8 7.5s9.1-3 10.8-7.5L128 160l56.5-21.2c4.5-1.7 7.5-6 7.5-10.8s-3-9.1-7.5-10.8L128 96 106.8 39.5C105.1 35 100.8 32 96 32s-9.1 3-10.8 7.5L64 96 7.5 117.2zm352 256c-4.5 1.7-7.5 6-7.5 10.8s3 9.1 7.5 10.8L416 416l21.2 56.5c1.7 4.5 6 7.5 10.8 7.5s9.1-3 10.8-7.5L480 416l56.5-21.2c4.5-1.7 7.5-6 7.5-10.8s-3-9.1-7.5-10.8L480 352l-21.2-56.5c-1.7-4.5-6-7.5-10.8-7.5s-9.1 3-10.8 7.5L416 352l-56.5 21.2z"/></svg>

Say I want to have multiple of these and I want them to have different sizes. And I have no framework for that. I might, of course, write JavaScript functions that create a SVG element, equip it with the right attributes and children, and use that throughout my code, like

// HTML part
<div class="magic-sparkles-container">
</div>

// JS part
for (const element of [...document.getElementsByClassName("magic-sparkles-container")]) {
    elements.innerHTML = createMagicSparkelsWand({size: 24});
}

// note that you need the array destructuring [...] to convert the HTMLCollection to an Array

// also note that the JS part would need to be global, and to be executed each time a "magic-sparkles-container" gets constructed again

But one of the main advantages of React’s JSX is that it can give you a smooth look on your components, especially when the components have quite speaking names. And what I ended up to have is way smoother to read in the HTML itself

// HTML part
<magic-sparkles></magic-sparkles>
<magic-sparkles size="64"></magic-sparkles>

// global JS part (somewhere top-level)
customElements.define("magic-sparkles", MagicSparklesIcon);

// JS class definition
class MagicSparklesIcon extends HTMLElement {
    connectedCallback() {
        // take "size" attribute with default 24px
        const size = this.getAttribute("size") || 24;
        const path = `<path d="M234.7..."/>`;
        this.innerHTML = `
            <svg
                xmlns="http://www.w3.org/2000/svg"
                viewBox="0 0 576 512"
                width="${size}"
                height="${size}"
            >
                ${path}
            </svg>
        `;
    }
}

The customElements.define needs to be defined very top-level, once, and this thing can be improved, e.g. by using shadowRoot() and by implementing the attributesChangedCallback etc. but this is good enough for a start. I will return to some refinements in upcoming blog posts, but if you’re interested in details, just go ahead and ask 🙂

I figured out that there are some attribute names that cause problems, that I haven’t really found documented much yet. Don’t call your attributes “value”, for example, this gave me one hard to solve conflict.

But other than that, this gave my No-Framework-Application quite a good start with readable code for re-usable icons.

To be continued…

In the end – should you actually go framework-less?

In short, I wouldn’t ever do it for any customer project. The above was a hobby experience, just to have went down that road for once, but it feels there’s not much to gain in avoiding all frameworks.

I didn’t even have hard constraints like “performance”, “bundle size” or “memory usage”, but even if I did, there’s post like Framework Overhead is bikeshedding that might question the notion that something like React is inherently slower.

And you pay for such “lightweightness” dearly, in terms of readability, understandibility, code duplication, trouble with code separation, type checks, violation of Single-Level-of-Abstraction; not to mention that you make it harder for your IDE to actually help you.

Don’t reinvent the wheel more often than necessary. Not for a customer that just wants his/her product soon.

But it can be fun, for a while.

Serving static files from a JAR with with Jetty

Web applications are often deployed into some kind of web server or web container like Wildfly, Tomcat, Apache or IIS. For single-instance services this can be overkill and serving requests can easily be done in-process without interfacing with some external web container or web server.

A proven and popular framework for an in-process web container and webserver is Eclipse Jetty for Java. For easy deployment and distribution you can build a single application archive containing everything: the static resources, web request handling, all your code and dependencies needed to run your application.

If you package your application that way there is one caveat when trying to serve static resources from this application archive. Let us have a look how to do it with Jetty.

Using the DefaultServlet for static resources on the file system

Jetty comes with a Servlet-Implementation for serving static resources out-of-the-box. It is called DefaultServlet and can be used like below:

Server server = new Server();
ServerConnector connector = new ServerConnector(server);
connector.setHost(listenAddress);
connector.setPort(listenPort);
server.addConnector(connector);
var context = new ServletContextHandler();
context.setContextPath("/");
var defaultServletHolder = context.addServlet(DefaultServlet.class, "/static/*");
defaultServletHolder.setInitParameter("resourceBase", "/var/www/static");
server.setHandler(context);
server.start();

This works great in the case of static resources available somewhere on the filesystem.

But how do we specify where the DefaultServlet can find the resources within our application archive?

Using the DefaultServlet for static in-JAR resources

The only thing that we need to change is the init-parameter called resourceBase from a normal file path to a path in the JAR. What does a path to the files inside a JAR look like and how do we construct it? It took me a while to figure it out, but here is what I came up with and it works perfectly in my use cases:

private String getResourceBase() throws MalformedURLException {
	URL resourceFile = getClass().getClassLoader().getResource("static/existing-file-inside-jar.txt");
    return new URL(Objects.requireNonNull(resourceFile).getProtocol(), resourceFile.getHost(), resourceFile.getPath()
        .substring(0, resourceFile.getPath().lastIndexOf("/")))
        .toString();
}

The method results in a string like jar:file:/path/to/application.jar!/static. Using such a path as the resourceBase for the DefaultServlet allows you to serve all the stuff from the /static directory inside your application (or any other) jar containing the class this method resides in.

A few notes on the code

Why don’t we just hardcode a path after the jar:-protocol? Well, the file path may chance in several scenarios:

  • Running the application on a different platform or operating system
  • Renaming the application archive – it could contain the version number for example…
  • Installing or copying the application archive to a different location on the file system

Using an existing resource and reusing the relevant parts of the URL-specification for the resource base directory solves all these issues because it is computed at runtime.

However, the code assumes that there is at least one resource available in the JAR and that its path is known at compile time.

In newer JDKs like 21 LTS constructing an URL using the constructor is deprecated but I did not bother to rewrite the code to use URI because of time constraints. That is left up to you or a future release…

As always I hope someone finds the code useful and drops a comment.

JavaScript – some less known Gems

I guess we can all agree that the most fun part of anything related to JavaScript is reading foreign JavaScript code 🙂

But while most of any hardness in understanding foreign (especially older) code lies in the every-year-fluctuations in common style – I also can’t really tell why I have this urge to change each “function” to a “const” declaration nowadays – once in a while you stumble across some feature, that is just too arcane.

Which means that it’s at least worth knowing about – after that, you can decide for yourself whether these enter your active vocabulary.

So these are some of my recent findings. Feel free to add.

Labelled Loops

JavaScript allows you to label any statement with a unique identifier. This is probably not a surprise for any Svelte developer (the $:… syntax is exactly that), but it is most useful in loops, because you can break or continue an outer loop using this:

// "outer" is the label here

outer: for (...) {
  for (...) {
    ...
    if (weAreDone()) break outer;
  }
}

It might have some old-school “GOTO” vibes indeed, and one can argue that in most cases there might be a more concise solution right around the corner, but especially if you have a tricky lookup algorithm, this might come handy one day.

Comma Operator

While I wondered for some years who on earth actually uses the Comma operator in C/C++, just a few weeks ago I found out that JavaScript actually has the same thing.

It allows you to execute some expression, ignoring it’s return value and directly execute the next statement in that same expression.

// (expr1, expr2) does evaluate expr1 and expr2 and return expr2.

const a = (b = 5, 3);
// a is now 3, but b = 5;

let a, b;
for (a = 0, b = 0; a + b < 3; a++, b++) { console.log(a, b) }
// output:
// 0 0
// 1 1

However, I would not advise using this thing ever. I mean, if you have some complicated expression where you decide just to do some step before evaluating some crucial other step – maybe it’s time to refactor the whole method.

void Operator

The void operator is like a special case of the comma operator in that it evaluates something and then returns undefined.

const a = calculateStuff(); // a might be whatever
const b = void calcaluteStuff(); // b is undefined

const c = (calculateStuff(), undefined); // c is identical to b, see above

This is e.g. an expression that I found while having to read some minified React code. So it might be of a certain use if you want to minify the number of letters in your code, but a more readable way would be just defining a function evaluating calculateStuff(), then returning without a return value.

However, the MDN web docs (referenced again here) give some real use cases of that operator, so if you are into that cryptic knowledge, go right ahead.

Bitwise NOT as a “found in List” check

Recently, I was weirded out by having to look at a list inclusion check in the likes of:

const list = ["a", "b"];
if (~list.indexOf("a")) {
  alert("found");
}

And this piece of code will actualy reach the alert, while it might leave you in wonders what the “~” is doing here. Unless you are used to very low-level bit arithmetic, in which case you’d directly recognize the bitwise NOT. It’s the operator that inverts every bit of a binary representation of a number to it’s opposite.

And the whole magic here lies in that for normal numbers (or BigInt), this is mathematically the same as

~a = -a - 1;

especially:
~-1 = 0
~0 = -1

and that for JavaScript, 0 is a falsy value while any other number is a truthy one. So this code above is used just to explicitly distinguish whether indexOf() returned “-1”, which it does if the object in question was not found.

So there you have it, but I’d rather use the way more readable

if (list.includes("a")) {...}
Bonus: console.log() styling

Now this does not really fit to the other operations above, but nevertheless I hear of people who did not know that before, so I’ll just drop it.

If there is any argument of console.log() starting with “%c”, it will take the next argument as a styling instruction instead of normally printing it. That is, the next argument needs to be a valid string that could also appear in a HTML stlye=”…” attribute, as

console.log("%cSo Big!", "font-size: 100pt; color: magenta");

Now, considering that console.log() is really only the most rudimentary way to output some statements for debugging (and one usually neglects the other ones like console.time(), console.timeEnd(), console.table()), this is not the next biggest thing that your imaginary Crypto Blockchain AI SaaS startup just needed, but it’s neverless good to know if you need some distinction in your logs.

Conclusion: Do whatever you like with that knowledge

While there are many things that one might love or hate about JavaScript – e.g. you might like the boolean-coercion via !! or you might hate the ??= or you might, with a mission, peddle generation expressions with function*/yield to your team – or or or… – there are always some more things that even after some years just look weird to my trained eye.

I guess it’s not completely wrong to thing that such expressions are occult for a specific reason in that there are only a handful of legit use cases for these, and forcing occult expression in a code that exceeds script size might cause some serious pain in the future.

But nevertheless, knowledge is power, so have a nice powerful evening.

Nonreligious Guidance for the JavaScript vs. TypeScript Debate

It’s always fun times when developers in the internet get heated over some discussion about their tool stack. One current case seems to be that some developers experienced their cases of “TypeScript is not giving me an adequate return of investment” and there are several articles which boil down to “I just don’t like it” – just google something along abandoning / ditching / dropping TypeScript and the resulting discussions on Reddit.

Now – bad news for anyone enjoying online arguments: Never has wisdom been reached by stating advantages and disregarding the disadvantages. I took some time to reflect, as several of my projects at Softwareschneiderei as well as my private ones use different tech stacks, to note the cases where I was happy about each choice of language, and to note where I wished to have the other one.

Of course, there is some kind of rule that if there are two quite similar things, most humans would pick one of these things to embrace and caress, the other one to hate with a passion and then insult each other’s intelligence. That is, of course, very helpful and generally awesome not productive to actually change anyone’s mind.

Now I would mostly suggest people to try to get used to TypeScript in order to have that tool at your hand. But I also have cherished the flexibility of JavaScript and seen the case where I would prefer it at least for the current state of that corresponding project.

Let me elaborate.

Quick Note: TypeScript is not really a superscript of JavaScript

This has to be state beforehand. It is to be said that if you write your TypeScript in a fashion where you “as any” your types at will, I would not call this by that name. Yes, the language allows doing so, but the choice of any language is also the choice of a certain mindset going with it. Several linting presets even disallow the explicit any. Which makes sense, because if you love the “as any”, you are not thinking TypeScript anyway.

Yes, doing it sparingly is rather a code smell then a red flag. But the mindset of TypeScript does not include the mindset of JavaScript as subset, therefore TypeScript can not be anything like a superscript of JavaScript.

When I would use TypeScript

So when was I most happy about TypeScript?

  • where I already had a rather clear model of my domain and then had to extend or change the current functionality.
  • When writing new methods that work with clear types, the support in knowing what these things are give you a real support in productivity
  • When my last episode of development was some considerable time ago, or was done by a different developer
  • When the smaller parts / submodules / … interface each other in a clear fashion and most development is focussed on only particular parts. Therefore, if an API changes for a particular reason, having to redesign your types avoids dangerous regressions that happen down the line.
  • Also, if you have a clear use case of many different similar types of data. If it is not clear from seeing an object (“oh, this is a house, not an animal” vs “oh, this is a house per se, not an offer for a house for sale”), the type hints alone will speed up your though processes.
  • … also, if you don’t have an IDE which does some type analysis anyway.
And when did I prefer plain JavaScript?
  • I experienced my largest annoyance with TypeScript in cases where our development aimed at clarifying its domain model itself -as in, very experimental stages in which it is more important to scaffold a basis for discussion. I.e. changes where not just renaming a field or changing its type, but a fundamentally updated understanding.
  • Interfacing large modules where data gets serialized in between anyways (e.g. server-client-interactions) – remember that TypeScript does not garant you real type safety. Any object still can still be what it likes to be. If I have to double check any content anyway, I rather do so without the extra boilerplate.
  • When doing lots of functional programming. TypeScript is just plain ugly when you pass function types as an argument and I have not yet seen the case where that really prevented any mistakes.
  • Mostly, when I do “library” code as opposed to “application” code, especially when you deal with many intermediate types. Your code can become bloated by verbose type definitions which do not contain any real value. The extra work of having to think up a name for these does not make one a hero then.
  • Especially in having to deal with Redux or some of the React querying / web request / caching libraries that aim to make your life easier etc. – sometimes these don’t even export all their types, being quite a hassle to write utility functions for them.

In short, forcing oneself to use TypeScript can lead to problems similar to the “wrong abstraction” problem. If you are in a state of development where you thoroughly define your types and these are (mostly smaller), clearly cut types, it’s likely that you gain traction by doing this work beforehand.

Conclusion: Don’t be too religious about it.

I consider it just not true that one cannot write large, safe projects in plain JavaScript. And one is still able to write monstrous, nonmaintainable projects with TypeScript. Sometimes the type definitions are just not the main concern in a current stage of development.

Think about it deliberately, and know each one’s advantages.

Also, some people currently propagate JSDoc as the current way most superior to all. I did not yet give it a proper chance, mostly because of its ugly aesthethics – but I’m open to trying it some day.

Addendum: JavaScript for flexible React Components

While this is a special case of my suggestion “rather use JavaScript for functional-programming-heavy cases”, you might run into TypeScript trouble a lot in cases where you want to use flexible React Components like

import ComponentA from ...;
import ComponentB from ...;

const FlexibleComponent = ({conditionProp, ...props}) => {

    const Component = React.useMemo(() =>
        conditionProp 
            ? ComponentA
            : ComponentB
        , [conditionProp]);

    return <Component {...props}/>;
};

While you can argue that usually this hints at “you need a better pattern for ComponentA and ComponentB, if they share so many similarities”, such a construct might be useful if patching together several external dependencies.

I have not yet found a way to cleanly match this distinction using TypeScript, especially since external dependencies might come – see above – with closed type definitions. Of course, you might go the “any” route here as well…

Converting a Grails app from war deployment in Tomcat to docker

A few years ago we had real servers with servlet containers installed and managed by administrators. These machines were bare-metal unicorns and had to be kept alive as long as possible (for many years). With the advent of first virtual machines and then container runtimes like Docker the approach and responsibilities for hosting web applications changed.

Our application not only went from Grails framework version 1 to 5 (atm, and soon to version 6) but also the above voyage regarding the hosting environment.

While the step from bare-metal to virtual machine was negligible from the developer and application side the migration to docker-based hosting is quite a step. Therefore I would like to depict the biggest changes of running a Grails application as a WAR deployment in Tomcat to a standalone application in Docker.

The original setting

We had our Grails application running on a server with a Tomcat servlet container for years. On the server we had also an ElasticSearch instance running and saved documents in a few well-defined directories on the local file system. Our database (Oracle in the past, PostgreSQL for over a year now) was running managed in a data center. We used container features like JNDI for parts of the configuration and datasources.

Deployment was essentially uploading a new WAR and context.xml file to the Tomcat servlet container using an Ansible playbook (where we restarted the servlet container to ensure not leaving any resource leaks…).

This setup worked quite well for years, only upgrades of the host operating system along with Tomcat and Java Virtual Machine (JVM) updates meant some work.

The target setup

Our customer has been in the process of converting most of the services running on virtual machines to dockerized deployments managed using Portainer. Most of the provisioning is automated and there is almost no snow flaking of the virtual machines hosting the docker runtimes. This eases the operation and administration of the services a lot and makes it much easier to setup new instances of a service and to monitor them.

So it was our task to migrate our setup on the the host VM to a docker-based deployment. In the future we basically push a docker image of our application to a docker registry of the customer and update the stack using Portainer.

The journey to a dockerized Grails app

The first thing to do were some changes to build.gradle to build a Jar-application instead of a WAR archive. We decided it was easier and more lightweight to run the Grails app standalone with the embedded tomcat than to use Tomcat in a docker container:

  • Remove the gradle war plugin
  • Change providedCompile dependencies to implementation
  • Add a task to prepare the Dockerfile and context to build our bootable application image:
task prepareDocker(type: Copy, dependsOn: assemble) {
    description = 'Copy files from src/main/docker and application jar to Docker temporal build directory'
    group = 'Docker'

    from 'src/main/docker'
    from project.bootJar

    into dockerBuildDir
}
  • Add an appropiate Dockerfile to our docker context in src/main/docker:
FROM eclipse-temurin:11-jdk-jammy
MAINTAINER Mihael Koep "mihael.koep@softwareschneiderei.de"

EXPOSE 8080

WORKDIR /app
COPY naomi-boot.jar .

ENV GRAILS_ENV development
# Additional env variables for configuration

CMD java -Dgrails.env=$GRAILS_ENV -jar /app/application-boot.jar
  • Remove JNDI usages and convert configuration from context.xml and JNDI to environment variables
  • Add a docker-compose.yml for the stack definition (here outlined for development with a database as part of the stack)
version: '3.7'
services:
  my-app:
    container_name: my-app-application
    image: ${IMAGE_TAG}
    environment:
      - GRAILS_ENV=development
      - DB_URL=jdbc:postgresql://my-app-db:5432/my-app
    volumes:
      - my-appdata:/app/data_home
    ports:
      - 8080:8080
    depends_on:
      - my-app-db
      - my-app-elastic
  my-app-db:
    container_name: my-app-database-stack
    image: postgres:13
    environment:
      - POSTGRES_USER=my-app
      - POSTGRES_PASSWORD=my-db-password
      - POSTGRES_DB=my-app
    ports:
      - 5432:5432
  my-app-elastic:
    container_name: my-app-elastic-stack
    image: docker.elastic.co/elasticsearch/elasticsearch:7.8.0
    environment:
      - node.name=es01
      - cluster.name=my-app-es
      - discovery.type=single-node
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - searchdata:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
      - 9300:9300
volumes:
  searchdata:
  my-appdata:

Conclusion

The journey from classical deployments to a dockerized environment using a docker stack is not that long for a Grails application (or most other Java web applications). It makes it trivial setting up a new instance or migrating it to another host as most configuration is in version controlled files and there is next to none snowflaking. The customer has an easier time running the web application because the requirements are only a container-runtime and explicitly formulated in the stack definition.

In addition, the developers are free to choose the JVM and other details within the containers without having to cross-check with the administrators of the customer if they are supported by their organization.

When custom React Hooks do not rerender Components on their own – make them.

Depending on who you ask, custom React Hooks are

  • a great way to stash away detailed inner workings of your application, making a) them reusable and b) your component cleaner and less complex
  • a horrible invention that hides away all the dreadful complexities one can think of, and by just making it invsible, not reducing any complexity at all

As usual, one has to calculate that balance depending on the use case, but in most cases I prefer my components to have a rather manageable lines-of-code-count (because this makes it easier to visually analyze their actual JSX structure, i.e. their semantics, what they are supposed to do.

However, sometimes an app grows over time and reaches a level of intricacy that seems to “outsmart” React itself, therefore breaking it. I do not know how to describe it otherwise:

I had a case of nested custom Hooks, in which one inner hook was executing a database query, giving a result and also a function to invalidate() and thus re-execute the query. It had been my understanding, that…

const useOurHook = () => {
    const query = useInnerHookWhichExecutesSomeQuery();

    console.log("query returned", query);

    return {
        result: query.result,
        invalidate: query.invalidate
    };
};

const Component1 = () => {
    const {result} = useOurHook();

    return <div>{JSON.stringify(result)}</div>;
};

const Component2 = () => {
    const {invalidate} = useOurHook();

    return (
        <button onClick={() => invalidate()}>
            invalidate
        </button>
    );
};

… pressing the button in Component2 will update the return value of the inner query hook, thus update the return value of the outer hook and finally update Component1.

However, that just did not happen. Even stranger, I could see my updated query result in the console.log statement within useOurHook(), but Component1 was staying as it was.

It took me several attempts in the inner workings of my both hooks, I tried to wrap the return values inside React.useMemo(), or to specifically put them inside a React.useState() that was explicitly set by a React.useEffect() – which should rather have the same outcome, but then again I do not know the actual React source code by heart – and there was just nothing that helped.

If you have any explanation for me that excels “yeah, React was broken” in its level of insight, please tell me. (maybe I have to read some docs, but it wasn’t obvious…)

So this is what helped. Rather than passing the invalidate function to my components, I decided to use the update functionality of the Redux useSelector() hook in such a way:

const useOurHook = () => {
    const lastRequestAt = useSelector(state => state.somewhere.lastRequestAt); // get timestamp from Redux store
    const query = useInnerHookWhichExecutesSomeQuery();

    React.useEffect(() => {
        if (lastRequestAt > 0) {
            query.invalidate();
        }
    }, [lastRequestAt, query.invalidate]);

    console.log("query returned", query);

    return {
        result: query.result,
    };
};

const Component1 = () => {
    const {result} = useOurHook();

    return <div>{JSON.stringify(result)}</div>;
};

const Component2 = () => {
    const dispatch = useDispatch();

    return (
        <button onClick={() => dispatch(updateRequest())}>
            invalidate
        </button>
    );
};

//////// and somewhere in a Redux slice:

...
reducers: {
    updateRequest: (state) => {
        state.lastRequestAt = Date.now();
    }
}
...

and this brought me the desired results. Now, I saw the update of query.result not only in the console.log, but also in Component1.

Now I agree that it appears quite wasteful to employ something as overbearing as Redux just to work around my weird situation, but I had Redux in my project anyway. I guess you couuld also use another state management or custom useContext() solution to work around this, just to give you an idea.

But I found it quite remarkable. It went against what I knew about React that you can have a hook update (visible in the console.log) without actually having React update a component that uses its return value.

Please, please – if any of you has any hint for insight, or is just curious about my concrete use case – I’ll be happy to discuss.

Interacting with SVG files inside your React applications

So, for some reason you have a SVG file that somehow resembles a part of the application you are currently developing.

This might be only a sketch that you want to prepare as a Click Dummy, or it might be that you need to display a somewhat unique, complicated structure that is best layed out per SVG editor. This is somewhat expected when you have customers in the technical / scientific research sector.

So now you want to fill it with life.

Now, the SVG format is quite close to the <svg> structure that one can embed into HTML, but there are some steps in between. Most importantly, most SVG Editors fill their .svg files up with meta data or specific information only required for the editor in case you want to edit the files again.

Thus, you have three choices to integrate the SVG component in a React application

  • Re-Build your SVG with custom React Components that, via JSX, render their <svg>, <g>, <path>, etc. accordingly
  • Convert your SVG to valid JSX – this is possible in many cases, but you need to take care e.g. that the style attribute is a string in the SVG and an object in JSX, also it can be still way too large to be readily put in a single React Component
  • Import your .svg as its own React Component and then wrap that into an own Component that takes care about the interaction part

While I also have written a small converter that does the SVG-JSX-Conversion just fine for me for any file that comes out of Inkscape (probably an idea for my next blog post), we had the case of some files with about 16000 lines of SVG each, so I chose the third method in our case.

In my eyes, it is very correct to mention that the following way somehow goes against the React Mindset. In which you render all your components yourself to attach them the required mouse event handlers, never having to interact with a HTML “id” or any document.getElementById() or document.getElementByClassName().

In any React application, these should be avoided, but the idea here is to have a singular point – a SvgWrapper Component – where you allow these functions because you’d agree about the need to somehow target the specific SVG elements.

The gist:

import {ReactComponent as OurHorrificSvgMonster} from "/src/monster.svg";

const OurBeautifulComponent = () => {

    useOurCarefulSvgSynchronizationEffect(); // more on this below

    return (
        <SvgWrapper>
             <OurHorrificSvgMonster/>
        </SvgWrapper>
    )
};

Quick note: you can target the embedded <svg> element itself by <OurHorrificSvgMonster ref={...}/> and you could use this (ref.current holds that HTML element) to traverse all the children, so if you know much about the structure of your svg you could even live without the <SvgWrapper>. But say someone else made the horrific svg monster and all you have is the id or class names to all the individual svg elements inside.

Then

const WithVanillaHandlersConnected = ({children}) => {
    const dispatch = useDispatch();

    React.useEffect(() => {

        const onClick = (event) => {
            dispatch(awesomeAction(event.target.id));
        };

        const awesomeElements = [...document.getElementsByClassName("awesome")];

        awesomeElements.forEach(elem => {
            elem.addEventListener("click", onClick);
        });

        return () => awesomeElements.forEach(pipe => {
            pipe.removeEventListener("click", onClick);
        });
    }, []);

    return children;
};

I chose this dispatch() as a placeholder for any interaction with the surrounding web application, it could also be a simple React state or something. You can register any event listener you want here (also “mouseover”, “mouseout”, “contextmenu”, …), but think of removing it again in the effect return function.

By the way, document.getElementsByClassName(…) returns something like a HTMLCollection which is not exactly iterable, thus the […destructuring] to make the .forEach() possible.

We now have the first part – our elements (in our case, everything that has class “awesome”) has got a click handler that allows to dispatch anything to the application state. But now they need to change, too.

In a purely React-y way, this could be done by a svg element that chooses its fill = {isActive? "magenta" : "black"} but as we choose not to render our components ourselves, we need to once again grab deeply into the DOM and dare to manipulate it by hand.

As mentioned above – this is a step towards very ugly problems as React cannot guarantee that your visual layer matches your application state. You, on yoru own, have to guarantuee to do what’s right.

This is where this comes in:

/*
 for this example, think of that the redux selector selectSomethingFromTheState returns something like:

result = [
  {elementId: "elem1", isActive: true},
  ...
];

and isActive could be the thing that was toggled by our awesomeAction() above

*/

const useOurCarefulSvgSynchronizationEffect = () => {
    const elementStates = useSelector(selectSomethingFromTheState);

    React.useEffect(() => {
        for (const state of elementStates) {
            const element = document.getElementById(state.elementId};
            element.style.fill = isActive ? "magenta" : "black";
            // ... do other stuff with the DOM element
        }
    }, [elementStates]);
};

There we have it – we have the SvgWrapper and the use…SynchronizationEffect() that both stray from the React mindset by accessing the DOM directly, but we do it in a fashion where it is concisely encapsulated inside <OurBeautifulComponent> and there is no direct knowledge about the IDs inside the SVG, or Class manipulations, CSS Selectors, etc. elsewhere.

In my opinion, one can indeed go against the rules if it’s necessary, but I also see the option for a Stockton Rush quotation here.. so, if you know of any more elegant way, please feel free to share.

PS: by the way, if you use vite, you might get an “Uncaught SyntaxError” when trying import { ReactComponent ... }I’ve written about this before.

Developing for Cordova + SQLite in a standard Browser environment

As any developer, who doesn’t just love it when a product that has grown over the years suddenly needs to target a new platform (e.g. operating system) because some customer demands changed, some dependency broke or some other totally unexpected thing called “progress” happened?

Fortunately, there are some approachs to cross-platform development and if one expects such a change of direction, one can early on adopt a suitable runtime environment such as Apache Cordova or Capacitor/Ionic or similar, who all promise you a Write-Once-Run-Anywhere experience, decoupling the application logic from the lower-level OS interactions.

Unfortunately though, this promise is a total lie and usually, after starting such a totally platform-agnostic project, really soon you will want to use a dependency that will only work for one platform and then your options are limited.

One such example is a Cordova project we are currently moving from Android to iOS, and in that process also redesigning a nice, modern frontend to replace a very outdated (read: unmaintainable) Vanilla JS application. So now we have set it up smoothly (React + Vite + Typescript – you name it!), so technically we do not need anything iOS-specific yet, so we can work on our redesign in a pure-browser environment with hot reloading and the likes – life is good!

Then comes the realization that our application is quite data heavy and uses an on-device SQL database to persist its data, and we don’t have that in the browser – so, life turned bad.

What to do? There had been a client-side WebSQL database specification once, but this was unofficial and never fully implemented, abandoned in 2010, still present in Chrome but they are even live announcing how they are removing it, so this is not the future-proof way to go.

We crave a smooth flow of development.

  • It is not an option to re-build the app at every change.
  • It is not an option to have the production system use its SQLite DB and the development environment to use a totally different one like IndexedDB – certain SQLite queries are too ingrained in our application.
  • It’s only probably an option to use an experimental technology like absurd-sql, which aims to fill in that gap but then again needs advanced API features like Web Workers, SharedArrayBuffer, Atomics API which we wouldn’t require else
  • It is possible to use in-memory SQLite via sql.js but for persistence, it wasn’t instantly obvious to me how to couple that with the partially supported Origin Private File System API

So after all, this is the easiest solution that still gave me most of my developer smoothness back: Use sql.js in memory and for development, display two nice buttons on the UI which let me download the whole DB and upload one from file again. This is the sketch:

We create a CombinedDatabase class which, depending on the environment, can hand out such a database in a Singleton-like manner

class CombinedDatabase {

    // This is the Singleton-part

    private static instance: CombinedDatabase;

    public static get = async (): Promise<CombinedDatabase> => {
        if (!this.instance) {
            const {db, type} = await this.createDatabase();
            this.instance = new CombinedDatabase(db, type);
        }
        return this.instance;
    };

    private static createDatabase = async () => {
        if (inProductionEnvironment()) {
            return {
                db: createCordovaSqliteInstance(),
                type: "CordovaSqlite"
             };
        } else {
            const sqlWasmUrl = (await import("../assets/sql-wasm.wasm?url")).default;
            // we extend the window object for reasons I tell you below
            window.sqlJs = await initSqlJs({locateFile: () => sqlWasmUrl});
            const db = new window.sqlJs.Database();
            return {db, type: "InMemory"};
        }
    }


    // This is the actual flesh, i.e. a switch of which API to use

    private readonly type: string;
    private cordovaSqliteDb: SQLitePlugin.Database | null = null;
    private inMemorySqlJsDb: SqlJsDatabase | null = null;

    private constructor(db: SQLitePlugin.Database | SqlJsDatabase, type: string) {
        this.type = type;
        switch(type) {
            case "CordovaSqlite":
                this.cordovaSqliteDb = db as SQLitePlugin.Database;
                break;
            case "InMemory":
                this.inMemorySqlJsDb = db as SqlJsDatabase;
                break;
            default:
                throw Error("Invalid CombinedDatabase type: " + type);
        }
    }

   // ... and then there are some methods

}

(This is simplified – in actual, type is an enum for me , and there’s also error handling, but you know – not the point here).

This structure is nice, because you can now implement low-level methods like some executeQuery(...) etc. which just decide depending on the type, which of the private DB instances it can address, and even if they work differently, return a unified response format.

The rest of our application does not know anything about any Cordova-SQLite-dependency, or sql.js, or whatever. Life is good again.

So How do Import / Export work?

I gave the CombinedDatabase some interfacing methods, similar to


    public async export() {
        switch (this.type) {
            case "CordovaSqlite":
                throw Error("Not implemented for cordova-sqlite database");
            case "InMemorySqlJs":
                return this.inMemorySqlJsDb!.export();
            default:
                throw Error("DB not initialized, cannot export.");
        }
    }

    public async import(binaryData: Uint8Array) {
        if (this.type !== CombinedDatabaseType.InMemorySqlJs) {
            throw Error("DB import only implemented for the in-memory/sql.js database, this is a DEVELOPMENT feature!");
        }
        await this.close();
        this.inMemorySqlJsDb = new window.sqlJs.Database(binaryData);
    }

This is also the reason why I monkey-patched the window object earlier, so I still have this API around outside the Singleton instantiation (createDatabase). Yes, this is a global variable and a kind of hack, but imo is what can safely be done inside the Browser within some good measure.

Remember, in Typescript you need to declare this e.g. in some global.d.ts file

import {SqlJsStatic} from "sql.js";

declare global {
    interface Window {
        sqlJs?: SqlJsStatic
    }
}

Or go around the Window interface by casting (window as any).sqlJs – you decide what you prefer.

Anyway, the export() functionality can then be used quite handily, it returns the in-memory database as a binary array and you can make the browser download that via a Blob URL:

api.db.export().then((array: Uint8Array) => {
    const blob = new Blob([array], {type: "application/x-sqlite3"});
    const link = document.createElement("a");
    link.href = URL.createObjectURL(blob);
    link.download = `bonpland${Date.now()}.db`;
    link.target = "_blank";
    link.click();
});

And similarly, you can use import() by reading a Uint8Array from a temporary <input type="file"> element with a FileReader() (somewhat common solution, but just comment below if you want the details).

To be exact, I don’t even use the import() button anymore because I pass my development DB as an asset to the dev server. This is nice (and only takes a few seconds on hot reloading because our DB is like 50 MB in size), but somewhat Vite-specific, which is why I will postpone this topic to some later blog time.