mirror of
https://github.com/vitorpamplona/amethyst.git
synced 2026-07-22 07:48:27 +00:00
docs(geode): SQLite knobs A/B verdict — no winner, knobs stay off by default
Backlog items 4-5 close. Both-variants-in-one-run A/B at 50k, repeated with relay order reversed: every delta flipped with the order (the second-running relay won queries and ingest latency in BOTH runs), so readers=8 / mmap_size=256MiB / temp_store=MEMORY / periodic PRAGMA optimize are all noise-level on container-class hardware. The config plumbing stays (hardware-dependent, operators should measure their own box); the example config now says so explicitly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TtDNpayEYvJH7QuPswND3A
This commit is contained in:
2
PLANS.md
2
PLANS.md
@@ -38,7 +38,7 @@ listing (including archived plans), open that folder's `README.md`.
|
||||
| cli | 6 | 5 | 1 | 0 | 0 | [cli/plans](cli/plans/README.md) |
|
||||
| quic | 4 | 3 | 0 | 0 | 1 | [quic/plans](quic/plans/README.md) |
|
||||
| quic/interop | 1 | 1 | 0 | 0 | 0 | [quic/interop/plans](quic/interop/plans/README.md) |
|
||||
| geode | 4 | 4 | 0 | 0 | 0 | [geode/plans](geode/plans/README.md) |
|
||||
| geode | 5 | 4 | 0 | 0 | 0 | [geode/plans](geode/plans/README.md) |
|
||||
| docs (frozen) | 52 | 48 | 2 | 0 | 2 | [docs/plans](docs/plans/README.md) |
|
||||
|
||||
## Live work (not shipped)
|
||||
|
||||
@@ -46,6 +46,11 @@ path = "/"
|
||||
in_memory = false
|
||||
file = "/var/lib/geode/events.db"
|
||||
|
||||
# NOTE: the four knobs below measured as pure noise in relayBench A/B
|
||||
# runs on 4-core container hardware at 50k events (see
|
||||
# geode/plans/2026-07-04-sqlite-knobs-ab.md) — their value is
|
||||
# hardware-dependent, so benchmark on YOUR box before enabling.
|
||||
#
|
||||
# Reader-connection pool size (file-backed stores only). Default 4.
|
||||
# readers = 4
|
||||
|
||||
|
||||
50
geode/plans/2026-07-04-sqlite-knobs-ab.md
Normal file
50
geode/plans/2026-07-04-sqlite-knobs-ab.md
Normal file
@@ -0,0 +1,50 @@
|
||||
# Server-side SQLite knobs — A/B verdict: no winner on container hardware
|
||||
|
||||
**Status: closed (negative result recorded).** Backlog items 4–5 of the
|
||||
relay performance campaign.
|
||||
|
||||
## What was tested
|
||||
|
||||
The knobs the campaign had previously reverted as noise-inconclusive,
|
||||
re-tested under the A/B protocol — both variants in ONE relayBench run
|
||||
(identical container conditions), then the run repeated with the relay
|
||||
order reversed to expose order bias. 50k synthetic corpus, all four
|
||||
knobs on the candidate at once:
|
||||
|
||||
```toml
|
||||
[database]
|
||||
readers = 8 # reader pool (default 4)
|
||||
mmap_size = 268435456 # 256 MiB
|
||||
temp_store_memory = true
|
||||
optimize_interval_seconds = 15 # PRAGMA optimize between ingest and queries
|
||||
```
|
||||
|
||||
## Result
|
||||
|
||||
Every observed delta flipped with run order. In BOTH runs the
|
||||
second-running relay won most query scenarios and ingest latency —
|
||||
regardless of which variant it was:
|
||||
|
||||
- run 1 (plain → knobs): knobs won 7/10 query scenarios, better ingest
|
||||
p99 (8.2 vs 16.0 ms).
|
||||
- run 2 (knobs → plain): plain won 6/10 query scenarios, better ingest
|
||||
p50/p99/throughput (8,552 vs 8,015 ev/s).
|
||||
|
||||
Storage size identical (±1 MiB). The periodic `PRAGMA optimize` also
|
||||
did not move the author-archive/planner-sensitive scenarios.
|
||||
|
||||
## Verdict
|
||||
|
||||
**All knobs stay off by default.** The `[database]` config plumbing
|
||||
ships anyway (readers / mmap_size / temp_store_memory /
|
||||
optimize_interval_seconds) because their value is hardware-dependent —
|
||||
an operator on NVMe with a large page cache or a memory-constrained VPS
|
||||
should measure on their own box — and `optimize_interval_seconds`
|
||||
remains sensible operationally for long-running relays even without a
|
||||
measurable win at 50k-fresh-database scale.
|
||||
|
||||
**Do not re-benchmark these on container-class 4-core hardware** without
|
||||
first fixing the run-order bias: the protocol note in
|
||||
`quartz/plans/2026-07-03-incremental-negentropy-storage.md` applies —
|
||||
same-run A/B plus order reversal is the minimum, and anything that
|
||||
doesn't survive the order flip is noise.
|
||||
@@ -1,6 +1,6 @@
|
||||
# geode plans
|
||||
|
||||
_Audited 2026-06-30. 4 plans: 4 shipped (archived), 0 in-progress, 0 queued, 0 abandoned._
|
||||
_Audited 2026-06-30. 5 plans: 4 shipped (archived), 0 in-progress, 0 queued, 1 closed (negative result)._
|
||||
|
||||
Performance-focused design docs for future work. Each file is a
|
||||
self-contained sketch — problem statement, observed numbers, proposed
|
||||
@@ -26,3 +26,9 @@ regressions show up in the regular CI matrix once they're enabled.
|
||||
- [archive/2026-05-07-live-broadcast-fanout-index.md](archive/2026-05-07-live-broadcast-fanout-index.md)
|
||||
- [archive/2026-05-07-connection-scaling.md](archive/2026-05-07-connection-scaling.md)
|
||||
- [archive/2026-05-07-negentropy-large-corpus.md](archive/2026-05-07-negentropy-large-corpus.md)
|
||||
|
||||
## Closed (negative result)
|
||||
|
||||
| Plan | Verdict |
|
||||
| ---- | ------- |
|
||||
| [2026-07-04-sqlite-knobs-ab.md](2026-07-04-sqlite-knobs-ab.md) | readers/mmap/temp_store/PRAGMA-optimize knobs: no winner on container hardware; every delta flipped with run order. Knobs ship config-gated, off by default. |
|
||||
|
||||
Reference in New Issue
Block a user