Category: General

The Number You See Is Not Necessarily the Number You Get

There is a particular class of bugs that is easy to dismiss as a floating-point problem.

For example you have a value in Excel: 25604.88

You open the spreadsheet, and that is exactly what you see. You read the cell with a library, however, and suddenly you get something like:

25604.880000000001

At first this looks like a broken spreadsheet, a broken XML file, or a broken library.

It is none of those.

An .xlsx file is a ZIP archive containing XML files. If you inspect the worksheet XML, you may encounter a value along the lines of:

<c r="A1">
    <v>25604.880000000001</v>
</c>

This is not Excel suddenly deciding that the user entered a different number.

The important distinction is that Excel displays numbers according to formatting and its rules for presenting floating-point values. The underlying numeric representation is based on binary floating point. Microsoft documents Excel’s use of IEEE 754 floating-point representation and the resulting precision limitations.

This distinction is worth understanding because it can turn a seemingly harmless value into a surprisingly difficult integration bug.

Why can’t a computer just store 25604.88?

Because computers don’t normally store floating-point numbers as decimal fractions.

A typical double uses the IEEE 754 binary64 representation. Conceptually, the value is represented using a sign, an exponent and a significand. There are 64 bits available in total, with 53 bits of precision in the significand.

This works beautifully for binary fractions.

For example:

0.5   = 1/2       = 0.1b
0.25  = 1/4       = 0.01b
0.75  = 1/2 + 1/4 = 0.11b
0.125 = 1/8       = 0.001b

All of these can be represented exactly in binary.

How to convert with simple Operations:

0.625 * 2 = 1.25 => 1
0.25  * 2 = 0.5  => 0
0.5   * 2 = 1    => 1
=> 0.101b

But other fractions can not be represented by binary fractions exactly like 0.1.

0.1 * 2 = 0,2 => 0
0.2 * 2 = 0.4 => 0
0.4 * 2 = 0.8 => 0
0.8 * 2 = 1.6 => 1
0.6 * 2 = 1.2 => 1 
0.2 * 2 = 0.4 => 0 // REPEAT
0.4 * 2 = 0.8 => 0
0.8 * 2 = 1.6 => 1
0.6 * 2 = 1.2 => 1
=> 0.00011001100110011001100110011...b

It is a unending number, limited by memory space for storing the number. When translating back to decimal is won’t be exactly 0.1 again.

0.00011001100110011001100110011b = 0.09999999962747097015d

Or for our example with 25604.88 the decimals 88 will be convert like this:

0.88 * 2 = 1.76 => 1
0.76 * 2 = 1.52 => 1
0.52 * 2 = 1.04 => 1
0.04 * 2 = 0.08 => 0
0.08 * 2 = 0.16 => 0 
0.16 * 2 = 0.32 => 0 
0.32 * 2 = 0.64 => 0
0.64 * 2 = 1.28 => 1
0.28 * 2 = 0.56 => 0
0.56 * 2 = 1.12 => 1
0.12 * 2 = 0.24 => 0
0.24 * 2 = 0.48 => 0
0.48 * 2 = 0.96 => 0
0.96 * 2 = 1.92 => 1
0.92 * 2 = 1.84 => 1
0.84 * 2 = 1.68 => 1
0.68 * 2 = 1.36 => 1
0.36 * 2 = 0.72 => 0
0.72 * 2 = 1.44 => 1
0.44 * 2 = 0.88 => 0 // REPEAT
=> 0.11100001010001111010...b

What does this mean for programmers?

The important lesson is we need to know what we are dealing with.

When reading numbers from external sources such as Excel, JSON or a database, don’t assume that the value you see is exactly the value your program receives. Check the actual value and the type your library produces.

For many calculations, this is completely fine. But when exact decimal values matter — for example, with money, accounting or other business rules — don’t leave this to chance.

Use a representation that matches the requirement. For example use BigDecimal or BigFraction for exacter decimal values.

You heard it here first: Stay clear of Always-Updating Software

Depending on who you ask, I would be described as one of the most laid-back, laissez-faire people around, or a dangerously megalomaniac, unrealiable weirdo [Author’s Note: this would lead us astray], and so in terms of what-software-to-use, I would consider most people as the world-leading expert in what works best for them personally.

However, one pattern seems to grind my gears, and while I don’t feel that the Schneide Blog is the best place for overbearing rants based on minor inconveniences, I still recognize a pattern that might be worrying because it is, at least, getting more and more common.

There is some software that is ALWAYS, whenever you start it, surprising you with a software update. Sometimes it is forced upon you, sometimes with sleazy sleights of hand, sometimes you even have the right to have a say in that matter, but nevertheless, it’s ALWAYS. Might not be numerical-always, but still noone on Polymarket would bet against you, so, it’s ALWAYS.

In case you would’t know what software I am talking about – I can give you several names from the top of my head – but you probably know lots of examples on your own 🙂

My core message is, that currently, these scream: “I will bring you harm!”

What is going on with software, having a need to constantly disturb my workflow – a thing that is holy for any serious software developer?
(a) maybe the border-radius of the third panel in the About… dialog was slightly off, and the application shutdown could be improved about 200-300µs.
(b) they are now relying purely on AI to write their most critical core features, with no sensitivity for security whatsoever.

And while I can totally understand (a), in current times you should be honest to endorse – unless you know better – it must be (b).

I, myself, am sometimes willing to recognize that I am not the most risk-averse human in the history of the world, maybe ever; and you probably really are more of an expert of what is good for you than my opinion would count; so, I also use some of these regularly. So, do your own what-could-possibly-go-wrong-gambles, but it is our job to at least feel smart about it when shit is going down (in hindsight, naturally).

Point is, I _could_ be bothered to check the release notes of any of these before any update. But because their move is “we demand on this urgency”, and my thought is “I’m in the process of saving my customer’s life (again)”, I will rarely do that, and unless they communicate better, I think there needs to be a rightful place to call these dangerously megalomaniac, unreliable weirdos out.

Dynamic Device Classes in Python

When writing PyTango device servers, it is common to implement one Python class per Tango device class. For small projects, this approach is simple and easy to understand. However, it becomes cumbersome when the set of available devices is not known at development time.

Consider a device server that should be entirely driven by a configuration file. Instead of hard-coding every supported device class, the server reads the device definitions at startup and creates the required Tango device classes automatically.

The goal is to write a generic PyTango server that does not need to be modified whenever a new device class is added. If a new entry appears in the configuration file, the server should simply create the corresponding Tango device class.

Python type

At first glance, this sounds unusual. After all, Python classes are typically defined using the familiar class keyword:

class Motor(Device):
    pass

Most Python developers stop here and never think about how classes are actually created. Under the hood, however, classes are objects themselves, and Python provides a built-in mechanism to construct them dynamically.

The function responsible for this is type().

Most of us use it in its simplest form to inspect the type of an object:

print(type(42))
# <class 'int'>

Less well known is its three-argument form:

type(name, bases, attributes)

The arguments are:

  • name: the name of the new class.
  • bases: a tuple of parent classes.
  • attributes: a dictionary containing class attributes and methods.

This means that the following definition is equivalent to the above class definition.

Motor = type(
    "Motor",
    (Device,),
    {},
)

The resulting object is exactly the same: a Python class that can be instantiated or registered with PyTango.

The third argument of type() becomes particularly interesting when more than just the class name should be configurable. The attributes dictionary allows methods, properties, or other class members to be added dynamically. This can be useful when Tango attributes or commands are also described in the configuration file.

Motor = type(
    "Motor",
    (Device,),
    {
        "some_property": 42,
    },
)

a = Motor()
print(a.some_property)
# 42

Conclusion

For our use case, creating classes dynamically becomes straightforward. Reading the configuration file and creating the required classes can be done in a simple loop. From PyTango’s perspective, there is no difference between a statically defined class and one created dynamically using type(). Both behave like ordinary Python classes.

Avoiding Code Style Discussions

Every developer has personal formatting preferences.
Brace placement, line wrapping, imports, tabs vs. spaces — everybody has an opinion, and most of them are reasonable.

The problem starts when all these styles meet in one repository.

The cost of “personal style”

A codebase written by ten developers can easily look like ten different applications stitched together. Suddenly, pull requests are full of formatting changes. Git diffs become noisy. Merge conflicts appear because one developer reformatted a file differently than another. Code reviews drift into discussions about whitespaces instead of actual functionality.

Even worse: inconsistent code slows down reading.

Humans recognize patterns quickly. When code follows the same visual structure everywhere, the brain spends less effort parsing syntax and more effort understanding intent.

Consistent formatting reduces cognitive load.

A shared style is less about aesthetics and more about reducing friction. But how to solve this problem?

Shared Project Style

In IDEs like IntelliJ, you can define a code style and automatically reformat code according to those rules. This helps you keep your own code consistent. However, if every developer uses a different style, it does not help the project as a whole.

You can configure the style under:

Settings -> Editor -> Code Style

and save it as a project-level configuration. IntelliJ will then create a codeStyles folder with XML files inside the .idea directory.

The solution for sharing one configuration across the whole project is to commit these files to Git. This way, every developer working on the project uses the same code style configuration.

The IDE can then help enforce the agreed style by reformatting code before commit or even automatically on save.


Consistency beats preference

The important thing is not finding the perfect style. The important thing is agreeing on one.

A consistent codebase is easier to read, easier to review, and easier to maintain. Pull requests become smaller and cleaner because they contain actual changes instead of formatting noise.

Good formatting should be boring and automatic. That leaves more time for discussions that actually matter.

Out of Memory when loading Records in Rails

Recently I ran into a problem that only showed up outside the development environment.

I had a small script that needed to iterate over all records in the database and load blobs.

Document.all.each do |doc|
  process(doc.blob)
end

With a small dataset everything worked as expected.
With production-sized data, however, the job was terminated by the runtime with an out-of-memory error.

This behaviour is not surprising once you look at what all.each actually does.

How all.each works

When calling all.each ActiveRecord is loading the complete result set into memory before the iteration starts.
For large tables this means that thousands or even millions of Ruby objects are instantiated at once.

If each record also references additional data — for example blobs, attachments, or associations — the memory usage grows quickly.

Loading Records with find_each

ActiveRecord provides find_each for exactly this scenario:

Document.find_each do |doc|
  process(doc.blob)
end

In contrast to each, this method does not load all records at once.
Instead, records are fetched in batches and yielded one by one.

Conceptually the process looks like this:

  1. Load a limited number of records
  2. Yield them to the block
  3. Discard them
  4. Load the next batch

By default, find_each loads records in batches of 1000.
The batch size can be configured:

Document.find_each(batch_size: 100) do |doc|
  process(doc.blob)
end

find_each always iterates in primary key order. This means the model must have a primary key that is orderable like integer or string. Any explicit ordering will be ignored.

If more control is required, find_in_batches can be used instead. It requires manual iteration over the batches.

Conclusion

Iterating over large tables with all.each is easy to write but can lead to excessive memory usage once the dataset grows.

For batch processing tasks, find_each is usually the safer default because it limits the number of instantiated records and keeps memory usage predictable.

Fighting the Paper War as a Team

Anyone who has ever gone through a public tender knows the feeling: forms on forms, references to other forms, appendices that depend on annexes, and fields that must be filled exactly as specified somewhere on page 37 of a different document. This is not a task; it is a paper war.

Trying to fight this war alone is a mistake.

We learned that the most effective way to survive such bureaucratic battles is to treat them like a team sport. Not a big team—three people are enough—but with clearly defined roles.

The Problem with the Lone Warrior

The naive approach is simple: one person sits down, opens all documents, and starts filling things out.

This person must:

  • understand the overall structure of the process,
  • search for the right documents and sections,
  • enter data correctly and consistently,
  • double-check everything afterward.

That is a lot of cognitive load. The result is usually slow progress, rising frustration, and errors that only show up when it’s already too late.

The paper war doesn’t reward heroics. It rewards coordination.

A Three-Person Setup

We had much better results by splitting the work into three distinct roles, all active at the same time.

1. The EXECUTOR

The executor is the only person who actually enters data into the forms.

This role is deliberately narrow:

  • type exactly what is agreed upon,
  • do not search,
  • do not interpret,
  • do not “improve” anything on the fly.

The executor’s job is flow. By removing all other responsibilities, they can focus on speed and accuracy.

2. The Navigator

The navigator owns the overview.

They know:

  • which document is relevant right now,
  • where a specific field is defined,
  • which appendix explains which requirement.

While the executor is typing, the navigator is already preparing the next reference: “Next field is in document B, section 4.2, and it depends on the value we used earlier in A.3.”

This prevents context switching for the executor and keeps the process moving forward.

3. The Checker

The checker validates everything live.

They verify:

  • numbers,
  • names,
  • dates,
  • consistency with previous entries,
  • alignment with external sources (contracts, invoices, registers).

This is crucial: checking after the fact is expensive. Checking while data is entered is cheap. Errors are caught immediately, while the context is still fresh.

Like a Car Driving Lesson

This setup is not unfamiliar if you think about a car driving lesson.

The executor is the driver. They focus entirely on operating the vehicle: steering, braking, accelerating. They don’t decide where to go next; they just execute cleanly and safely.

The navigator is the driving instructor sitting in the passenger seat. They know the route, anticipate upcoming turns, and give timely instructions so the driver can react without stress.

The checker plays the role of the driving examiner in the back seat. Quiet but attentive, they observe everything, immediately spotting mistakes, inconsistencies, or rule violations before they become real problems.

Just like in a driving lesson, separating these roles creates confidence, flow, and control—exactly what you need when navigating bureaucratic traffic.

Why This Works

This setup mirrors patterns we already know from software development:

  • separation of concerns,
  • reducing cognitive load,
  • fast feedback loops.

Each person has a clear responsibility, and overlaps are intentional but limited. Nobody is idle, and nobody is overwhelmed.

Most importantly, the process becomes predictable. Instead of a chaotic scramble through documents, you get a steady, almost mechanical flow from field to field.

Paper Wars Won’t Disappear

Bureaucratic processes are unlikely to become simpler anytime soon. Digital forms often just move the paper war onto a screen without changing its nature.

But how we approach them can change.

Treating a public tender as a collaborative, real-time effort instead of a solitary endurance test turns frustration into something manageable—and sometimes even efficient.

You may not win the war forever.
But at least you’ll win this battle.

Splitting a repository while preserving history

Monorepos or “collection repositories” tend to grow over time. At some point, a part of them deserves its own life: independent deployments, a dedicated team, or separate release cycles.

The tricky part is obvious: How do you split out a subproject without losing its Git history?

The answer is a powerful tool called git-filter-repo.

Step 1: Clone the Repository into a New Directory

Do not work in your existing checkout.
Instead, clone the repository into a fresh directory by running the following commands in Git Bash:

git clone ssh://git@github.com/project/Collection.git
cd Collection

We avoid working directly on origin and create a temporary branch:

git checkout -b split

This provides a safety net while rewriting history.

Step 2: Filter the Repository

Now comes the crucial step. Using git-filter-repo, we keep only the desired path and move it to the repository root.

python -m git_filter_repo \
  --path path/to/my/subproject/ \
  --path-rename path/to/my/subproject/: \
  --force
  • --path defines what should remain
  • --path-rename moves the directory to the repository root
  • --force is required because history is rewritten

After this step, the repository contains only the former subdirectory — with its full Git history intact.

Step 3: Push to new repository

Now point the repository to its new remote location:

git remote add origin ssh://git@github.com/project/NewProject.git

If an origin already exists, remove it first:

git remote remove origin

Rename the working branch to main:

git branch -m split main

Finally, push the rewritten history to the new repository:

git push -u origin main

That’s it — the new repository is ready, complete with a clean and meaningful history.

Conclusion

git-filter-repo makes it possible to split repositories precisely. Instead of copying files and losing context, you preserve history — which is invaluable for git blame, audits, and understanding how the code evolved.

When refactoring at repository level, history is not baggage. It’s documentation.

Happy splitting!

Oracle and the materialized view update

Materialized views are powerful. They give us precomputed, queryable snapshots of expensive joins and aggregations. But the moment you start layering other views on top of them, you enter tricky territory.

The Scenario

You define a materialized view to speed up a reporting query. Soon after, others discover it and start building new views on top of it. The structure spreads.

Now imagine: you need to extend the base materialized view. Maybe add a column, or adjust its definition. That’s when the trouble starts.

The Problem

Unlike regular views, materialized views don’t offer a convenient CREATE OR REPLACE. You can’t just adjust the definition in place. Oracle also doesn’t allow a simple ALTER to add a column or tweak the structure—recreating the materialized views is often the only option.

Things get even more complicated when other views depend on your materialized view. In that case, Oracle won’t even let you drop it. Instead, you’re greeted with an error about dependent objects, leaving you stuck in a dependency lock-in.

The more dependencies there are, the more brittle the setup becomes. What started as a performance optimization can lock you into a rigid structure that resists change.

As a short example, let’s look at how other databases handle this scenario. In Postgres, you can drop a materialized view even if other views depend on it. The dependent views temporarily lose their base and will fail if queried, but you won’t get an error on the drop. Once you recreate the materialized view with the same name and structure, the dependent views automatically start working again.

What to Do?

That is the hard question. Sometimes you can try to hide materialized views behind stable views. Or you take the SQL of all dependent views, drop them, change the materialized view, and then recreate all dependent views— a process that can be a huge pain.

How do you manage changes to materialized views that already have dependent views stacked on top? Do you design around it, fight with rebuild scripts every time, or have another solution?

The Dimensions of Navigation in Eclipse

Following up on “The Dimensions of Navigation in Object-Oriented Code” this post explores how Eclipse, one of the most mature IDEs for Java development, supports navigating across different dimensions of code: hierarchy, behavior, validation and utilities.

Let’s walk through these dimensions and see how Eclipse helps us travel through code with precision.

1. Hierarchy Navigation

Hierarchy navigation reveals the structure of code through inheritance, interfaces and abstract classes.

  • Open Type Hierarchy (F4):
    Select a class or interface, then press F4. This opens a dedicated view that shows both the supertype and subtype hierarchies.
  • Quick Type Hierarchy (Ctrl + T):
    When your cursor is on a type (like a class, interface name), this shortcut brings up a popover showing where it fits in the hierarchy—without disrupting your current layout.
  • Open Implementation (Ctrl + T on method):
    Especially useful when dealing with interfaces or abstract methods, this shortcut lists all concrete implementations of the selected method.

2. Behavioral Navigation

Behavioral navigation tells you what methods call what, and how data flows through the application.

  • Open Declaration (F3 or Ctrl + Click):
    When your cursor is on a method call, pressing F3 or pressing Ctrl and click on the method jumps directly to its definition.
  • Call Hierarchy (Ctrl + Alt + H):
    This is a powerful tool that opens a tree view showing all callers and callees of a given method. You can expand both directions to get a full picture of where your method fits in the system’s behavior.
  • Search Usages in Project (Ctrl + Shift + G):
    Find where a method, field, or class is used across your entire project. This complements call hierarchy by offering a flat list of usages.

3. Validation Navigation

Validation navigation is the movement between your business logic and its corresponding tests. Eclipse doesn’t support this navigation out of the box. However, the MoreUnit plugin adds clickable icons next to classes and tests, allowing you to switch between them easily.

4. Utility Navigation

This is a collection of additional navigation features and productivity shortcuts.

  • Quick Outline (Ctrl + O):
    Pops up a quick structure view of the current class. Start typing a method name to jump straight to it.
  • Search in All Files (Ctrl + H):
    The search dialog allows you to search across projects, file types, or working sets.
  • Content Assist (Ctrl + Space):
    This is Eclipse’s autocomplete—offering method suggestions, parameter hints, and even auto-imports.
  • Generate Code (Alt + Shift + S):
    Use this to bring up the “Source” menu, which allows you to generate constructors, getters/setters, toString(), or even delegate methods.
  • Format Code (Ctrl + Shift + F):
    Helps you clean up messy files or align unfamiliar code to your formatting preferences.
  • Organize Imports (Ctrl + Shift + O):
    Automatically removes unused imports and adds any missing ones based on what’s used in the file.
  • Markers View (Window → Show View → Markers):
    Shows compiler warnings, TODOs, and FIXME comments—helps prioritize navigation through unfinished or problematic code.

Eclipse Navigation Cheat Sheet

ActionShortcut / Location
Open Type HierarchyF4
Quick Type HierarchyCtrl + T
Open ImplementationCtrl + T (on method)
Open DeclarationF3 or Ctrl + Click
Call HierarchyCtrl + Alt + H
Search UsagesCtrl + Shift + G
MoreUnit SwitchMoreUnit Plugin
Quick OutlineCtrl + O
Search in All FilesCtrl + H
Content AssistCtrl + Space
Generate CodeAlt + Shift + S
Format CodeCtrl + Shift + F
Organize ImportsCtrl + Shift + O
Markers ViewWindow → Show View → Markers

The Dimensions of Navigation in Object-Oriented Code

One powerful aspects of modern software development is how we move through our code. In object-oriented programming (OOP), understanding relationships between classes, interfaces, methods, and tests is important. But it is not just about reading code; it is about navigating it effectively.

This article explores the key movement dimensions that help developers work efficiently within OOP codebases. These dimensions are not specific to any tool but reflect the conceptual paths developers regularly take to understand and evolve code.

1. Hierarchy Navigation: From Parent to Subtype and Back

In object-oriented systems, inheritance and interfaces create hierarchies. One essential navigation dimension allows us to move upward to a superclass or interface, and downward to a subclass or implementing class.

This dimension is valuable because:

  • Moving up let us understand general contracts or abstract logic that governs behavior across many classes.
  • Moving down help us see specific implementations and how abstract behavior is concretely realized.

This help us maintain a clear overview of where we are within the hierarchy.

2. Behavioral Navigation: From Calls to Definitions and Back

Another important movement is between where methods are defined and where they are used. This is less about structure and more about behavior—how the system flows during execution.

Understanding this movement helps developers:

  • Trace logic through the system from the point of use to its implementation.
  • Identify which parts of the system rely on a particular method or class.
  • Assess how a change to a method might ripple through the codebase.

This navigation is useful when debugging, refactoring, or working in unfamiliar code.

3. Validation Navigation: Between Code and its Tests

Writing automated tests is a fundamental part of software development. Tests are more than just safety nets—they also serve as valuable guides for understanding and verifying how code is intended to behave. Navigating between a class and its corresponding test forms another important dimension.

This movement enables developers to:

  • Quickly validate behavior after making changes.
  • Understand how a class is intended to be used by seeing how it is tested.
  • Improve or add new tests based on recent changes.

Tight integration between code and test supports confident and iterative development, especially in test-driven workflows.

4. Utility Navigation: Supporting Movements that Boost Productivity

Beyond the main three dimensions, there are several supporting movements that contribute to developer efficiency:

  • Searching across the codebase to find any occurrence of a class, method, or term.
  • Generating boilerplate code, like constructors or property accessors, to reduce repetitive work.
  • Code formatting and cleanup, which helps maintain consistency and readability.
  • Autocompletion, which reduces cognitive load and accelerates writing.

These actions do not directly reflect code relationships but enhance how smoothly we can move within and around the code, keeping us focused on solving problems rather than managing structure.

Conclusion: Movement is Understanding

In object-oriented systems, navigating through your codebase along different dimensions provides essential insight for understanding, debugging, and improving your software.

Mastering these dimensions transforms your workflow from reactive to intuitive, allowing you to see code not just as static text, but as a living system you can navigate, shape, and grow.

In an upcoming post, I will take the movement dimensions discussed here and show how they are practically supported in IDEs like Eclipse and IntelliJ IDEA.