About

inorderly.com is the home of Inorderly, a small Go engine for deterministic event ordering. It takes events that arrive late, twice or out of order and releases them as one sequence. Give it the same events again, in any order and with any repeats, and it writes the same sequence, byte for byte.

Why It Exists

Events rarely arrive the way they happened. A webhook provider retries, so an order shows up twice. Two services stamp events with clocks that disagree, so a payment looks older than the order it pays for. A feed handler sees tick 88 before tick 87. Most teams patch this with a dedupe table, a sort buffer and a timeout, and the patch mostly works.

The trouble comes when the events are run again: for a backfill, an audit, a bug hunt, or a second system that has to agree with the first. The hand-made patch often gives a slightly different sequence the second time, and nobody can say why. Inorderly makes the sequence a fixed function of the events, with the rules written down, so a replay gives the same answer and you can prove it did.

How It’s Built

  • A bounded window. Events are held until the newest event time has moved a set distance past them. An event that arrives later than that is refused as late, and reported, rather than slipped in somewhere new.
  • A published order. Time, then the counter of a hybrid logical clock, then producer, sequence number and id. The same rule every time.
  • Parents first. An event can name the events it depends on. It waits for them, for a bounded time, and goes straight after them even when its clock is behind theirs. Parents that never come are named in the output.
  • Duplicates dropped, conflicts refused. The same event twice is one event. Two different events claiming one identity are reported, never merged quietly.
  • A replay hash. The SHA-256 of the output. Two runs compare in one line, or with sha256sum.
  • Crash-safe runs. With checkpoints on, a run killed at any moment carries on from where it was and writes exactly what one straight run would have.
  • Any language. The inorderly command reads and writes NDJSON, so any producer can feed it. Go programs can embed the library.
  • Tested by shuffling. The tests feed the same events in thousands of arrival orders and kill the process mid-run, and check the output never changes. The results are published with the code.

License

All the code is open source under the Apache License 2.0, on GitHub. You can use it, change it and build it into your own products.

Questions People Ask

Is it really free?

Yes. Inorderly is free to use, and the code is free to take under the Apache License 2.0.

Does it need a server or a database?

No. It’s one binary, or one Go package, with nothing else to install. Checkpoints are plain files in a folder you choose.

Kafka keeps order within a partition, but not across partitions, and it can’t tell that two messages are the same event sent twice by an upstream retry. Flink and Beam can order events with watermarks, as part of a stream processing engine you deploy and run. Inorderly is just the ordering step, as a library or a command, with the rules written down and a hash to check them. It can sit in front of anything, including a Kafka consumer.

What happens to an event that comes too late?

It’s refused, counted, and written to the rejects file if you keep one. It never changes the output of the events that were on time, which is what keeps that output repeatable.

Do my producers need special clocks?

No. Any integer time works, such as milliseconds since 1970. Producers that use a hybrid logical clock, like the hlc package that comes with Inorderly, give every child a timestamp above its parents’, so nothing ever has to wait.

Does it scale to many machines?

Version 0.1 orders one stream in one process. To order more, run one Inorderly per stream, keyed the way your data already splits.

Can I suggest something?

Please do. The contact page has the address.