Verderer — a provable record of how public data changes
Turns "did this dataset quietly change?" into something you can prove: every observation is attested in an append-only log, and a downloadable bundle verifies offline against independent anchors.
Correctness under complexity, pointed at public trust. Verderer continuously observes high-value U.S. government environmental pages and datasets, cryptographically attests every observation in a tamper-evident ledger, detects meaningful change (definition swaps, threshold edits, silent dataset shifts — not byte noise), and emits a public, citable, self-verifying record.
The problem
An archive is only as trustworthy as the archiver. "Trust us, this is what the page said" is not proof — the operator (or the source) could alter history. The question worth answering isn't "do you have a copy?" but "can a stranger verify the copy is honest, without trusting you?"
Approach — a provable trust spine
- Rust Merkle log (C2SP
tlog-tiles— the transparency-log design behind Certificate Transparency and Go's checksum database) with signed checkpoints published as static tile files, so a verifier recomputes inclusion proofs with no live service. - Independent time anchors — RFC 3161 timestamps from third-party TSAs, roots pinned in the verifier, checked offline.
- Witness cosignatures with quorum verification — no single party (including Verderer) can silently rewrite the log.
- Consistency-proof gossip —
verify-consistencyproves offline, under a pinned key, that a later checkpoint extends an earlier one, so a forked, shrunk, or rewritten log is caught. - A downloadable proof bundle that verifies entirely in the browser (WebAssembly) or offline via the Rust verifier.
Why it's trustworthy — the point is verifiability, not authority
The whole design refuses to ask for trust. Corrupt any stored leaf and verify
reports INVALID; the offline verifier confirms a record against a signed
checkpoint trusting neither the source nor Verderer. Change detection spans
static pages, JS-rendered tools, and scientific/tabular datasets
(CSV/JSON, NetCDF/HDF, zip/xlsx) — down to cross-unit numerics, so 10 ppb
equals 0.010 ppm instead of raising a false alarm. Collection is polite by
construction (robots.txt, per-host rate limits, conditional GET), and the
verifier is open source on purpose — the proofs only mean anything if anyone can
audit the checker.
What's technically interesting
Applying transparency-log engineering — the same tamper-evidence tech that secures the web's certificate ecosystem — to civic data integrity, across a Rust trust kernel, a Python collection/diff pipeline, and an Astro public record with in-browser WASM verification. Every observation is also archived as a standard WARC, so the record interoperates with the volunteer rescue ecosystem (Wayback, End-of-Term, EDGI) instead of being self-referential. Status: the core roadmap (M0–M8) is complete and confirmed, and the "real tool" arc is underway — it runs itself on a schedule, and the differ and WARC reader are Hypothesis-fuzzed against untrusted bytes, with inclusion/consistency proofs scale-tested at 100k leaves. Deferred deliberately: Bitcoin-anchored timestamps stay off the trust path until a real confirmed fixture can be verified — no synthetic anchors.