The claims ledger · Wave 5 + Clean Sweep · July 2026

Every claim. Every loss. On the record.

The system of record for AI keeps a record of itself. Each row: verdict, figure, condition, date, engine SHA, and a command you can run from the published harness. Benchmarks run in numbered waves; the 2026-07-12 cloud re-lock put 3-run error bars on the standing ratios. The next wave re-runs the full set — verdicts may change; the dates and SHAs never do.

ledger chain head af7852f70cdbddd3… — sha-256 over every row's (id · date · SHA), enforced at build: a changed date or SHA on any past row fails the deploy. Verdicts may change; provenance never does.

The AgentDB claims ledger, Wave 5 + Clean Sweep · July 2026: every performance claim with verdict, figure, conditions, and verification date.
Axis	Verdict	Figure · comparator	Condition & caveats	Verified
Multi-modal atomicity	UNIQUE	one transaction relational + vector + full-text + graph + audit in one commit, one snapshot	proven kill-9 durable (Wave 1, 2026-07) — the only engine demonstrating this $ cargo test -p agentsql-engine --test kill9_multimodal -- --ignored	2026-07-01 e56a61f last verified 2026-07-07
Agent-native governance	UNIQUE	shipped eval-gated deploy, exact answer replay, public-key-verifiable audit	engine primitives, not app conventions — no other database expresses them $ DEPLOY PROMPT … REQUIRE EVAL … PASSED # runs in AgentDB Studio today	2026-07-07 e56a61f last verified 2026-07-07
Durable single-row writes (RPO=0)	LEADING	2.6–2.8× / 6.7–10× / 5.3–6× vs RocksDB / Fjall / SQLite synchronous=FULL — every single-row write leg	fsync-before-ack RPO=0 on every arm (--sync; AgentDB Strict) · fresh Hetzner ccx33, fstrim'd · 3 runs, mean ± stdev ≤6.6% · AgentDB 28.9–30.1k singles/s ~parity with Postgres sync-commit=on (0.94–1.18× — Postgres pays a localhost hop, inherent to its architecture). Batch legs ≥0.83× the disk-KV tier, ahead of the SQL tier throughout. An initial pass with the adapter still at Relaxed was caught and re-run — only the Strict arms are published as durable. $ crud-bench -d <engine> -s 100000 -c 12 -t 24 -r --skip-scans --skip-indexes --sync # ×3, bench/cloud_relock_2026-07-12/	2026-07-12 ac1f1dd last verified 2026-07-12
Point reads (query-surface engines)	LEADING	29.2× / 19.4× point reads vs SQLite / Postgres — 1.71M ops/s, fastest system with a query surface	same 3-run durable matrix, fresh ccx33 · read 1.71M ±5.7% ops/s Bare mmap/LSM KV stores still win pure point reads — see the 0.71–0.74× row below. Stated both ways. $ crud-bench -d <engine> -s 100000 -c 12 -t 24 -r --skip-scans --skip-indexes --sync # ×3	2026-07-12 ac1f1dd last verified 2026-07-12
Embedded TPC-C at RPO=0 (single warehouse)	LEADING	252,045 tpmC · 3.42× vs SQLite WAL+FULL at matched RPO=0 — p99.9 65× better (24.1 vs 1,567 ms)	TPC-C subset, warehouses=1, terminals=10, 3×30s, seed=42 · Strict fsync-before-ack · 252,045 ± 34,237 vs 73,779 ± 1,777 · 12 consistency invariants PASS, 0 errors Statistical tie with Fjall on raw defaults (1.17×, overlapping error bars) with p99 3× better. DuckDB 19×, SurrealDB 59× behind. $ cargo run --release -p agentsql-bench --features bench-sqlite,bench-duckdb,bench-fjall,bench-surrealdb -- tpcc --warehouses=1 --terminals=10 --duration=30s --runs=3 --seed=42	2026-07-08 98d9b94 last verified 2026-07-12
Vector ANN query (SIFT-1M, real corpus)	LEADING	6.8× · tie query p50 6.8× ahead of pgvector · statistical tie with Qdrant's stable runs	1M vectors, 10k queries, HNSW matched (M=48, ef=500) · recall@10 0.9988, all three systems within 0.0006 · p50 1.53 ms vs pgvector 10.34 ms · QPS 659 vs 99 · 3 interleaved runs No Qdrant win claimed — its mean and best run still edge ahead (p50 1.34 ms mean incl. one fast outlier; stable runs 1.47–1.50 ms, bars overlap with 1.53). Both comparators are servers paying a localhost hop; AgentDB runs embedded — stated. $ cargo run --release -p agentsql-bench --features bench-postgres,bench-qdrant -- vector-scale --dataset=sift1m:<dir>/sift --runs=3 --k=10 --systems=agentsql,pgvector,qdrant	2026-07-09 87666ee last verified 2026-07-12
Full-text BM25 ranked top-k	LEADING	3.8× ranked top-10 QPS @1t vs SQLite FTS5 (p50 67 µs vs 252 µs = 3.7×)	100k docs, 20k-word Zipf vocab, same BM25 constants both sides · ranking agreement 0.982 Jaccard · build ~parity (1.13× behind incl. optimize) · 3 runs, ratios ±2% @8t measured 61× (FTS5's concurrent arm collapses below its own 1t rate); even granting FTS5 ideal 8× scaling, 2.75× ahead. Engine gap, filed: explicit ORDER BY bm25() falls off the indexed path today — the `@@ … LIMIT` shape is the ranked query. $ cargo run --release -p agentsql-bench --features bench-sqlite --example w2d_bm25_leg # ×3	2026-07-11 dc0ad36 last verified 2026-07-12
Graph depth-3 traversal	LEADING	8× / ~70× depth-3 BFS QPS @8t vs a tuned recursive CTE · p50 vs SurrealDB embedded	100k nodes, 500k edges, out-degree 5 · p50 111–113 µs (1.24× ahead of the CTE even @1t) · result-set agreement Jaccard 1.0000 vs the CTE oracle · 2 runs, ratios ±3% Edge ingest is 22× behind raw batched SQLite inserts into a bare 2-column table (a statement-layer cost, the W2-F singles wall) while 4.1× ahead of SurrealDB. SurrealDB's recursive syntax was adopted by oracle validation, not parse success. $ cargo run --release -p agentsql-bench --features "bench-sqlite bench-surrealdb" --example w2d_traverse_leg	2026-07-12 d10413f last verified 2026-07-12
Ranked array top-k (tags + recency)	LEADING	33–80× CONTAINS + ORDER BY created_at LIMIT 10 vs Postgres GIN + btree	500k docs × 1–8 tags, 20k vocab · 8.6–10.7 µs vs 342–718 µs p50 — the composite array index serves a true top-k early-exit; Postgres has no ordered GIN, so it materialises the set then sorts · 3 passes ×2 runs No result cache rides this path (checked); repeats hit only the plan cache, same as Pg prepared statements. $ cargo run --release -p agentsql-bench --features bench-postgres --example w2d_array_leg	2026-07-12 d10413f last verified 2026-07-12
Unbounded array membership (full set)	LEADING	2.2–2.9× ahead of Postgres GIN on full-set CONTAINS at 500k docs (PK-covered postings vs GIN + heap fetch)	supersedes the leg-3 1.8×-behind cell (R5a): 85% of that query was materialising rows a PK-only projection never reads — the covering lane answers straight from exact-at-pin postings; GIN has no index-only scan so Pg always bitmap-heap-fetches · both measurement windows published (loaded box 2.9×, quieter window 2.22×, margin vs Pg's same-run arm each time) · membership agreement Jaccard 1.0000 every run The flip is for PK-projection membership — the exact shape leg 3 measured. Non-PK projections still hydrate and stay ~1.3–1.4× behind Pg at 500k: the owned-MVCC-row-materialization floor, stated as a trade below. $ cargo run --release -p agentsql-bench --features bench-postgres --example w2d_array_leg # + r5_array_membership_probe for attribution stages	2026-07-13 3ac76f33 last verified 2026-07-13
AS-OF reads under write load (time-travel throughput)	LEADING	≥62× AS-OF point-read QPS at a pinned fork vs Dolt — p99 ~600–850× better (6.6–9.3 µs vs 5.6–5.7 ms)	100k rows · 8 readers pinned at the fork while 4 durable writer threads accrete ~25–70k versions past the pin · 635–731k vs 10.0–10.3k QPS · concurrent durable writes 5–14× ahead · Strict RPO=0, Dolt fsync strace-verified · 5 runs Dolt pays a localhost MySQL round-trip (~50–100 µs floor — under 2% of its 5.6 ms p99, so the gap is engine-side; stated anyway). One 171k-QPS outlier run under ambient box load reported. $ cargo run --release -p agentsql-bench --features bench-dolt --example w2d_asof_throughput_leg	2026-07-12 d10413f last verified 2026-07-12
Durable OLTP throughput (TPC-C)	LEADING	10.69× vs SQLite synchronous=FULL	durability matched — RPO=0 both sides, fsync-before-ack · datacenter NVMe (Hetzner ccx33) · 22 warehouses, 22 terminals On laptop NVMe (cheap fsync) the same protocol reads 3.85× — the moat widens where fsync is real. AgentDB tpm 363,048 ±1.7% vs SQLite-FULL 33,967. $ cargo test -p agentsql-bench --features bench-sqlite,bench-postgres --release --test throughput_durability_matched_t3 -- --ignored --nocapture	2026-07-04 c52131a last verified 2026-07-04
Durable OLTP throughput (TPC-C)	LEADING	4.43× vs Postgres 16, sync_commit=on	durability matched — RPO=0 both sides · same box (Hetzner ccx33) · 22 warehouses, 22 terminals Postgres runs client/server — every statement pays a localhost TCP hop, inherent to its architecture; stated on every ratio. $ cargo test -p agentsql-bench --features bench-sqlite,bench-postgres --release --test throughput_durability_matched_t3 -- --ignored --nocapture	2026-07-04 c52131a last verified 2026-07-04
Durable OLTP throughput (TPC-C)	LEADING	3.85× / 5.11× vs SQLite-FULL / Postgres-SyncOn	durability matched, RPO=0 · consumer laptop NVMe (Core Ultra 7 155H) · 22 warehouses — AgentDB 642,595 tpm ±6.1% $ cargo test -p agentsql-bench --features bench-sqlite,bench-postgres --release --test throughput_durability_matched_t3 -- --ignored --nocapture	2026-07-03 14f25d3 last verified 2026-07-03
Durable OLTP tail latency (p99.9)	LEADING	20.7 ms vs 4,174 ms AgentDB p99.9 vs SQLite-FULL p99.9 (Postgres: 77.7 ms)	same durability-matched TPC-C matrix, ccx33 datacenter NVMe, 22t — SQLite pays a per-commit fsync stall; AgentDB amortises it in the group commit $ cargo test -p agentsql-bench --features bench-sqlite,bench-postgres --release --test throughput_durability_matched_t3 -- --ignored --nocapture	2026-07-04 c52131a last verified 2026-07-04
AS OF point reads (branch time-travel)	LEADING	~16× AS OF point-read p50 vs Dolt 2.1.10 — the only other database with branches + SQL AS OF	10k rows, 1,000 branches, RPO=0 both arms, same box, arms interleaved · AgentDB 0.003 ms vs Dolt 0.052 ms p50 AgentDB runs embedded; Dolt is a server paying a localhost hop (~20–40 µs floor). Subtracting the hop still leaves ~2–4× on warm runs — both numbers stated. $ cargo run --release -p agentsql-bench --features bench-dolt -- branchasof --rows=10000 --branches=1000 --reads=1000 --writes-per-branch=10 --seed=42 --systems=agentsql,dolt	2026-07-07 3a02f7e last verified 2026-07-07
Branch creates	LEADING	1.38× branch creates/s vs Dolt 2.1.10	563/s vs 407/s, 1,000 branches, RPO=0 both arms, same box Dolt's run 1 is a cold server; warm-only margin is 1.13× — both stated. $ cargo run --release -p agentsql-bench --features bench-dolt -- branchasof --rows=10000 --branches=1000 --reads=1000 --writes-per-branch=10 --seed=42 --systems=agentsql,dolt	2026-07-07 3a02f7e last verified 2026-07-07
Per-branch durable writes	LEADING	1.45× 10-UPDATE durable batch p50 vs Dolt 2.1.10	8.00 ms vs 11.63 ms per batch, RPO=0 both arms (Dolt fsync strace-verified), same box Embedded vs server hop applies here too, stated. $ cargo run --release -p agentsql-bench --features bench-dolt -- branchasof --rows=10000 --branches=1000 --reads=1000 --writes-per-branch=10 --seed=42 --systems=agentsql,dolt	2026-07-07 3a02f7e last verified 2026-07-07
Vector index build (SIFT-1M)	COMPETITIVE	2.4–3.0× · 1.55–1.97× behind build wall-clock ahead of pgvector · Qdrant keeps the lead (1.55× same-session, 1.97× quiet-box)	parallel HNSW build 117–141 s at recall 0.9988 — 14.8× its own single-threaded arm on 24 cores · pgvector 338–348 s · Qdrant 66–72 s Qdrant leads this sub-axis and we say so. Closed attributed-and-stated (R5b, 2026-07-13): one measured lane attempt (kernel ILP + prefetch + visited-scratch) came back neutral and is banked unmerged; the residual is memory latency plus Qdrant's segmented per-segment builds against our one MVCC/AS-OF-serving graph — a one-time build cost. $ cargo run --release -p agentsql-bench --features bench-postgres,bench-qdrant -- vector-scale --dataset=sift1m:<dir>/sift --runs=3 --k=10 --systems=agentsql,pgvector,qdrant	2026-07-09 787608c last verified 2026-07-13
Point reads (bare mmap/LSM KV tier) deliberate, permanent trade for transactional semantics — LMDB exists for a reason and we print the number	BEHIND	0.71–0.74× vs LMDB / Fjall zero-copy point reads — the KV tier keeps this axis	same 3-run ccx33 matrix, both durability axes · MVCC owned-row materialization vs zero-copy borrowed reads, with no SQL, no planner, no transactions on the other side $ crud-bench -d <engine> -s 100000 -c 12 -t 24 -r --skip-scans --skip-indexes # ×3	2026-07-12 ac1f1dd last verified 2026-07-12
Async single-row writes (relaxed durability) closed as attributed (W2-F) — the same discipline pays 2.6–2.8× ahead at matched RPO=0	BEHIND	0.24–0.37× vs RocksDB single-row create/update/delete at sync=false	matched-async axis of the same 3-run ccx33 matrix (AgentDB Relaxed: flusher still fsyncs, ack is page-cache) · the price of per-commit WAL discipline — assemble+CRC+chain under a commit fence vs a raw memtable append Still ahead of Fjall/LMDB/SQLite/Postgres on the same async singles (1.2–4.4×); the gap is to RocksDB's raw append only. $ crud-bench -d <engine> -s 100000 -c 12 -t 24 -r --skip-scans --skip-indexes # ×3	2026-07-12 ac1f1dd last verified 2026-07-12
Single-warehouse hot-row TPC-C vs no-fsync defaults the documented durability + optimistic-SI trade — the RPO=0 pairing is the publication result	BEHIND	2.32× behind SQLite WAL+NORMAL raw defaults (no fsync per commit) on one hot warehouse	same leg-7 cell, engines at their out-of-the-box durability — the no-fsync engines keep the raw-defaults crown. AgentDB-Strict now runs at 89% of its own Relaxed arm; at matched RPO=0 the verdict flips to 3.42× ahead (row above) $ cargo run --release -p agentsql-bench --features bench-sqlite,bench-duckdb,bench-fjall,bench-surrealdb -- tpcc --warehouses=1 --terminals=10 --duration=30s --runs=3 --seed=42	2026-07-08 98d9b94 last verified 2026-07-12
OLAP aggregation planned: vectorized executor	BEHIND	behind columnar aggregation vs DuckDB (DuckDB leads)	DuckDB's vectorized executor wins analytical scans today $ cargo run --release -p agentsql-bench --features bench-duckdb -- tpcc --systems=agentsql,duckdb	2026-07-07 5633312 last verified 2026-07-07
Durable writes (crud-bench, 9 competitors) superseded as a headline — kept as data	LEADING	5.2× creates/s vs SurrealDB's flagship RocksDB backend — their tool, their workload	durability on for every arm (--sync) · 100k rows, 12 clients, 24 threads · ccx33 · AgentDB 97,876 vs 18,705 creates/s. Also: 6.0× Postgres, 29.8× MySQL, ~97× SQLite-FULL Raw fsync-per-txn KV stores (RocksDB/LMDB/ReDB/Fjall) collapse to ~1–3k writes/s here — they have no group commit. That is the durable-write story. $ crud-bench -d <engine> -s 100000 -c 12 -t 24 -r --sync # SurrealDB's tool + the AgentDB adapter	2026-07-04 c52131a last verified 2026-07-04
Durable reads (crud-bench) superseded as a headline — kept as data	LEADING	763,081 ops/s 11.6–19.7× Postgres, MySQL, MongoDB, SurrealDB, SQLite	same durable crud-bench matrix, ccx33 · single fastest system with a query surface Bare memory-mapped KV stores (Fjall, LMDB, RocksDB) win pure point-reads by <2× — with no SQL, no MVCC, no planner — while losing durable writes 30–100×. Stated both ways. $ crud-bench -d <engine> -s 100000 -c 12 -t 24 -r --sync	2026-07-04 c52131a last verified 2026-07-04
Batch update/delete ×1000 superseded as a headline — kept as data	LEADING	3.07× / 3.55× batch update / batch delete ×1000 vs SurrealDB 3.0.5	same laptop, same harness, sync=never both sides (relaxed durability, labeled), scans excluded Relaxed-durability leg, kept for continuity; the flagship axis is the durability-matched matrix above. $ ./run-both-clean.sh # -s 100000 -c 12 -t 24 -r --skip-scans	2026-06-12 9461d19 last verified 2026-07-07
Paged DiskANN resident memory superseded as a headline — kept as data	LEADING	32–48× less resident vector data vs in-RAM HNSW	synthetic 16-latent-factor set, small N (≤4000); ratio is a representation property by construction Superseded as a headline by the real-corpus SIFT-1M result above; kept as data. $ cargo test -p agentsql-vector --release --test diskann_vs_hnsw_bench -- --ignored --nocapture	2026-06-10 9461d19 last verified 2026-07-07
Vector ANN query (SIFT-1M, real corpus) superseded as a headline — kept as data	COMPETITIVE	4.5× query p50 + QPS vs pgvector 0.8.4	1M vectors, 10k queries, HNSW params matched (M=48, ef=500) · recall@10 0.9986 vs 0.9992 · p50 2.40 ms vs 10.86 ms pgvector is a server paying a localhost hop; AgentDB runs embedded and this harness does not persist the index — both asymmetries stated in the ledger. $ cargo run --release -p agentsql-bench --features bench-postgres,bench-qdrant -- vector-scale --dataset=sift1m:<dir>/sift --runs=3 --k=10 --systems=agentsql,pgvector,qdrant	2026-07-07 220f09a last verified 2026-07-07
Unbounded array membership (full set) superseded as a headline — kept as data CLOSED by R5(a) (2026-07-13): PK-covered CONTAINS shipped — flipped to 2.2–2.9× ahead	BEHIND	1.8× behind Postgres GIN on full-set CONTAINS at 500k docs (429 vs 235 µs p50)	the measured crossover: at 50k docs AgentDB led 2.7–3.1×; at 500k the mid-band posting lists are ~10× longer and GIN + heap fetch materialises the full set faster · ingest ~parity, index build 1.4× ahead · membership agreement Jaccard 1.0000 $ cargo run --release -p agentsql-bench --features bench-postgres --example w2d_array_leg	2026-07-12 d10413f last verified 2026-07-13
Vector ANN query (SIFT-1M) superseded as a headline — kept as data CLOSED by W1-C (2026-07-09): the contiguous vector arena shipped — now a statistical tie	BEHIND	~1.3× behind query p50/QPS vs Qdrant 1.15.5 (Qdrant leads)	same SIFT-1M protocol · ~1.5× on warm quiet-box runs · recall within 0.0007 Qdrant wins this axis and pays a localhost REST hop doing it. AgentDB's index serves MVCC AS OF reads; Qdrant's structurally cannot. $ cargo run --release -p agentsql-bench --features bench-postgres,bench-qdrant -- vector-scale --dataset=sift1m:<dir>/sift --runs=3 --k=10 --systems=agentsql,pgvector,qdrant	2026-07-07 220f09a last verified 2026-07-07
Vector index build (SIFT-1M) superseded as a headline — kept as data CLOSED by W1-A (2026-07-09): parallel HNSW build shipped — 2.4–3.0× ahead of pgvector	BEHIND	4.4× / 14× behind build wall-clock vs pgvector / Qdrant (both lead)	AgentDB inserts on one thread under MVCC transactions; comparators run parallel batch builders. Per-thread rates are at parity (~500 vs ~457 vec/s·worker) $ cargo run --release -p agentsql-bench --features bench-postgres,bench-qdrant -- vector-scale --dataset=sift1m:<dir>/sift --runs=3 --k=10 --systems=agentsql,pgvector,qdrant	2026-07-07 220f09a last verified 2026-07-07
Single-warehouse contended TPC-C (w=1) superseded as a headline — kept as data documented optimistic-SI trade; multi-warehouse regime is the publication result	BEHIND	19.3× / 9.9× behind vs SQLite / Fjall on one hot row (they lead)	warehouses=1 contention microbench: single-writer engines serialize on one lock while AgentDB runs 10 optimistic terminals into one hot key — the documented optimistic-SI trade. At 22 warehouses durability-matched, the verdict flips to the LEADING rows above $ cargo run --release -p agentsql-bench --features bench-sqlite,bench-duckdb,bench-fjall,bench-surrealdb -- tpcc --warehouses=1 --terminals=10 --duration=30s --runs=3 --seed=42	2026-07-07 5633312 last verified 2026-07-07
Full-text search head-to-head superseded as a headline — kept as data	PENDING	— no published numbers	head-to-head not yet run; no claims made	2026-07-07 — last verified 2026-07-07
Graph traversal head-to-head superseded as a headline — kept as data	PENDING	— no published numbers	head-to-head not yet run; no claims made	2026-07-07 — last verified 2026-07-07

Method: every race is same-machine, same-harness, with durability modes disclosed on both sides and comparators at their best-shot configs. Ledger source: bench/RESULTS.md in the product repo; harness, configs, and competitor pins are public at github.com/meshbergio/agentdb-bench-harness. Don’t take the table’s word for it — book 30 minutes and pick the machine.