Platform · Archive
SPEC 02–03Points are acknowledged at the edge before anything else happens.
Acknowledgement starts at the edge. Write-ahead logs, store-and-forward through interrupted links, idempotent delivery and explicit gaps make the resilience model inspectable.
Collectors
SPEC 02 §4OPC UA and MQTT Sparkplug B, spoken natively.
The OPC UA collector subscribes with configurable sampling and deadband, reconnects with backoff, and records the disconnection — a gap in the data is evidence, not embarrassment. The MQTT collector implements Sparkplug B properly: birth and death certificates, aliases, sequence numbers, state management. Both run at the edge, next to the source, because that is where durability has to start.
Running a unified namespace? The Sparkplug collector subscribes to your broker’s tree and the topic paths become source URIs — the UNS keeps answering “what is it now”, and the historian remembers everything else.
state is illustrative. users, streams and collectors are not metered in self-hosted OrdinateDB.
Store-and-forward
SPEC 02 §6The write-ahead log acknowledges first. The network is somebody else’s problem.
Every collector writes each point to a local write-ahead log before acknowledging it to the source. If the uplink drops — for a minute or a fortnight — points accumulate at the edge and drain when the path returns, deduplicated on merge so a retry can never double-count. This is not a recovery mode; it is the only write path there is.
Evaluation should exercise the same failure path: interrupt the link, observe the local backlog, restore the path and reconcile acknowledged identities at the receiver. The mechanism and its evidence remain visible throughout.
this trace explains the path; it is not a published performance or recovery result.
Storage tiers
SPEC 03 §2Hot for the control room, warm for the engineers, cold for the decades — and cold is documented Parquet.
Recent data sits in a hot tier optimised for the queries a live display makes. As it ages it moves to a warm columnar tier, and eventually to cold storage: Apache Parquet files in a layout that is documented in the spec, on disk or object storage you control.
The cold tier is readable by DuckDB, Spark and pandas with no OrdinateDB process running. Tier movement changes where a point lives, never what it says — queries span all three tiers transparently, and the result does not depend on which tier answered.
Lifecycle policies
SPEC 03 §6No silent lossy compression. Ever. If data is thinned, you can read the decision.
Raw is the default, indefinitely. Where retention genuinely requires thinning, someone writes a lifecycle policy: a named, versioned, audited object that states what is kept, at what resolution, for how long, and who approved it. Every query over a thinned range says so in the result — the provenance travels with the number.
keep:
- raw for 180d # as written
- mean(10s) for 5y # declared
- parquet cold thereafter # open format
approved_by: m.okafor 2026-05-02
supersedes: v2 (raw 90d) — change audited
effect: queries over thinned ranges
carry lossy_ranges > 0If you are moving an existing archive off a proprietary historian, the migration path is documented end to end.