Efficiency and cost
Fewer bucket requests, less memory, and the s3bench harness over upstream denoland/celld.
Patches that cut bucket request counts, memory, and latency — with the
measurements to prove them, re-measurable in tools/s3bench/. Each
entry names its fork patch number. Bucket-posture group commit has its
own Fleet section.
Fewer bucket requests
0054— Eviction skips visibility LISTs it can prove locally (see Storage). Measured on 200 short-lived Durable Objects (6 writes each,CELLO_DURABILITY=bucket): 12 Class-A requests per cell before, 9 after.0055— A lone fleet node slows its posture-repair scan: while amaintainpass finds no eligible peer, the wait before the next pass doubles from one second up to the 30 s maintenance tick. The flat retry cost a lone node about 2.6M LIST plus 2.6M GET a month doing nothing. A degraded ensemble that lost all its followers still retries at once, so it re-recruits the moment one returns.0056— Concurrent bucket-posture writes on one cell share ownership reads (see Storage). Measured on one hot cell with 16 concurrent writers: GETcells/*/own.json1550 → 553 at the same throughput.0057— Eviction deletes the superseded per-cell L0/L1 objects behind the handoff snapshot (see Storage). Measured on the 200-agent shape: storedcells/bytes 2.47 MB → 0.57 MB, objects 1800 → 400.0061— Warm eviction guards HEAD instead of LIST (see Storage): one Class-B HEAD replaces one Class-A LIST.0071— Idle bundle GC goes quiet (see Fleet): an idle bucket node costs one discovery LIST and then nothing until it writes, instead of one LIST per 30 s tick (~2880 Class-A a day). Also fixes the bucket hot-cell double-write (per-cell L0s beside every bundle flush).0078— Lazy handoff for short chains: an eviction skips the full-database handoff snapshot when the cell’s restorable chain was born in this epoch, every acked row is already per-cell, and the chain holds at most 16 objects (HANDOFF_CHAIN_MAX_OBJECTS) — a successor restores the short L0 chain instead, a bounded number of Class-B GETs replacing one Class-A PUT plus a full-database upload per eviction. Modelled on ClickHouse MergeTree’s lazy merges; cells restored from an older epoch and longer chains still snapshot. Measured (200 agents × 6 writes, bucket, release builds): Class A 1934 → 1734 (−10%), write p50 unchanged at 210 ms.
Less memory and faster starts
0065— The sampler tick returns glibc free pages pastCELLO_C_TRIM_MB(default 8, was a 32 MiB const). Measured post-drain RSS 72.0 → 61.9 MiB on the agent-50 s3bench shape.0066—CELLO_SQLITE_CACHE_KBcaps the per-connection SQLite page cache (default 2000, the stock ceiling: no behavior change).0067— A deployment generation projects its/bundletree once instead of once per isolate; read-only semantics unchanged.0102— Hydration counting is O(1) (see Storage): an 8.9 GB whale’s fill no longer walks the whole cut per faulted read.
Measure it yourself
0058— A reproducible S3 request-cost harness intools/s3bench/:proxy.pylogs one JSON line per request in front of the S3 endpoint,count.pyclassifies the log intoMETHOD x key-familycounts with an R2 Class A/B summary,stored.pyinventories storedcells/objects and bytes,app/is a minimal Worker + SQLite Durable Object app, andrun.shdrives the agent workload (or one hot cell withHOT=1) against a local RustFS container. New files only; nothing undertools/runs in CI.0069— The s3bench agent workload drives its agents concurrently — one thread and connection per agent on its own phase — so arrivals look like independent clients and cross-cell group commit can batch. The serial walk kept at most one cell in flight and could never batch.0072— The s3bench drivers report per-turn latency (mean,p50,p95,p99, nearest-rank, whole ms) next to the request counts, so ack-latency regressions show beside the savings.0076—run.shdefaultsCELLO_BINtotarget/release/celloand refuses to start while anothercello-s3bench-*container is up (unlessS3BENCH_ALLOW_PARALLEL=1), after debug-vs-release and dueling-harness skews polluted two.11measurements.