On the 3rd of the month, the monthly production report goes out. On the 5th, a site that spent two days on a degraded link finishes draining its store-and-forward backlog, and the archive now contains points for the reporting period that the report never saw. On the 9th, the lab returns a moisture result for a sample taken on the 28th, and a shift supervisor corrects a manual reading that was fat-fingered on the 30th.
Every number in the report is now, in some small way, out of date. The report has already been sent. What should the archive do?
Late data is not an edge case
It is worth being clear that none of the above is anomalous. In plant data, lateness is structural:
- Store-and-forward drains. A collector that buffers through an outage is doing exactly what it should — see how the archive handles this — and by design it delivers old points at new times.
- Lab results. A sample has a validity timestamp of when it was drawn from the process, and an arrival timestamp of whenever the LIMS interface got round to it. Days apart, routinely.
- Manual corrections. Someone typed 71.2 when the gauge said 17.2. The correction is right; it is also late by definition.
An archive that pretends data arrives in order, once, and correctly the first time, is an archive that has to lie somewhere. The usual place it lies is history.
Two timelines
The established answer — bitemporal modelling, standard in serious financial systems for decades and largely absent from historians — is to admit that every value has two timestamps that answer different questions:
- Valid time: when the value was true in the process. The sample was drawn at 08:00 on the 28th.
- Transaction time: when the archive learnt about it. The result arrived at 14:31 on the 9th.
Traditional historians keep only valid time. When the lab result lands, it is inserted at its process timestamp as though it had always been there; when the correction lands, it overwrites. The archive is thereby made retroactively omniscient — and the report you sent on the 3rd becomes unreproducible. Query the same month again and you get different numbers with no explanation, which in an audit reads less like diligence and more like tampering.
As-of queries
Keep both timelines and the awkward questions become ordinary queries. OrdinateDB's AS OF clause fixes the transaction-time horizon — show me the archive as it stood at a given moment:
-- what the monthly report actually saw
SELECT avg(value), pct_good
FROM series('site4/FI-1102')
WHERE valid_time IN month('2026-04')
AS OF '2026-05-03T09:00:00Z';
-- what we now believe to be true
SELECT avg(value), pct_good
FROM series('site4/FI-1102')
WHERE valid_time IN month('2026-04');
The first query reproduces, exactly and permanently, the numbers that went out on the 3rd. The second gives today's best knowledge. And because both are just queries, the difference between them is too:
SELECT * FROM revisions('site4/FI-1102',
valid => month('2026-04'),
since => '2026-05-03T09:00:00Z');
-- every point that arrived or changed after the report,
-- with who or what changed it, and why
That diff is the correction notice to head office. Not "the numbers moved, trust us" — a listing of precisely which values changed, from what to what, on which authority.
Corrections never overwrite
This falls out of the model, but it is worth stating as a rule, because in regulated environments it is the whole point: a correction is a new fact about an old time, not a replacement of the old fact. The original value, its origin, the corrected value, the correcting identity and the correction time are all permanent records. GxP auditors have a term for systems where a value can be changed without trace, and plants that have been through a 21 CFR Part 11 inspection do not need the term explained. An overwrite is an erasure; an append is an audit trail.
What this costs
The reflexive objection is storage: two timestamps per point, versions kept forever — surely the archive doubles. It does not, for a boring reason: almost every point is written once and never revised. OrdinateDB stores transaction time per ingest batch, not per point, so the common case costs a few bytes amortised over thousands of points, and delta encoding eats most of that. Revisions are stored as append-only overlays keyed by the valid-time range they amend — you pay per correction, and corrections are (one hopes) rare. On real plant workloads the bitemporal overhead is low single-digit percent. The mechanics are specified in SPEC 03 §7, and the cold tier keeps both timelines in the same documented Parquet layout as everything else.
The report is still true
Bitemporality is usually presented as machinery. It is better understood as manners: it lets the archive be honest about the difference between the process and its own knowledge of the process.
The report you sent on the 3rd was not wrong. It was a correct statement of everything the archive knew on the 3rd, and with the transaction timeline kept, it remains one — reproducible to the point, forever. What changed afterwards is not the past; it is what you know about the past. An archive should be able to tell you both, and refuse to confuse them.