vole

Clone

Read only, no credentials:

https://rodent.software/vole.git

Write over SSH with a registered key:

git clone ssh://git@rodent.software:2222/vole.git

A free and open-source forge that lives in your Git repo.

Phase 0: measure metadata writes, git transport, and tree serving

c5626f1b87084870524ea54987ef47a65ea4740b · Keenan Nemetz · 18d ago · parent 17311880

Phase 0: measure metadata writes, git transport, and tree serving

Three throwaway spikes under docs/spikes, each with its own harness,
README, and reproduce command. Findings are folded into the roadmap and
its decision log.

- ref-cas: sharded + batched refs sustain 25,637 entity writes/s at 20k
  entities with zero lost writes; a single unsharded ref manages 30/s
  with 22 CAS retries per write and decays quadratically as it fills.
  Metadata refs must be sharded. Unsharded is now off the table.
- smart-http: driving git http-backend as a subprocess is correct (hook
  stderr reaches the client, aborts leave no ref and no orphaned
  children) and pushes 150 MiB/s on compressible data, but a 1 GiB push
  peaks at 1.13 GB of git-child RSS. Phase 1 needs a push size cap and
  a concurrency semaphore, plus chunked-upload spooling.
- tree-serving: sites can be served straight from git trees with no
  export step - 1.2 ms p50 cold, 0.075 ms warm, 170k req/s behind an
  18.5 MiB cache.

Harness bugs caught during the work are recorded in the spike READMEs,
including one that silently dropped 83% of writes while reporting
plausible throughput.

docs/spikes/README.md +30

@@ -0,0 +1,30 @@
	1	# Spikes
	2
	3	Throwaway experiments that settle a question before the design depends on it.
	4	Each has its own `main.go` and a README with method, measurements, and the
	5	decision it produced. Nothing here ships; the code exists to be re-run.
	6
	7	\| Spike \| Question \| Verdict \|
	8	\|---\|---\|---\|
	9	\| [ref-cas](ref-cas/README.md) \| Can metadata be commits on sidecar refs, and at what rate? \| Yes. Sharded + batched: 6,966/s at 2k entities, 25,637/s at 20k. A single ref is 30/s and decays. \|
	10	\| [smart-http](smart-http/README.md) \| Is `git http-backend` as a subprocess correct and fast enough? \| Yes. Hook messages propagate, aborts are clean. 1 GiB push peaks 1.13 GB RSS — needs a size cap. \|
	11	\| [tree-serving](tree-serving/README.md) \| Can sites be served straight from git trees? \| Yes. Cold 1.2 ms p50 from git; 170k req/s warm with an 18 MiB cache. No export step needed. \|
	12
	13	Run any of them with:
	14
	15	```sh
	16	go run ./ref-cas -n 2000
	17	go run ./smart-http -sizemb 1024
	18	go run ./tree-serving -files 10000 -conc 16 -reqs 4000
	19	```
	20
	21	Shared conventions worth keeping if these are ever rewritten:
	22
	23	- Verify, don't trust the throughput number. Every spike ends with a pass
	24	that counts what actually landed in the repo. That check caught a harness bug
	25	that silently dropped 83% of writes in ref-cas.
	26	- A scenario that can't fail is not a test. Two smart-http scenarios
	27	initially reused objects already on the server, so nothing transferred and the
	28	numbers were meaningless. Push to a fresh repo when measuring transfer.
	29	- Label measured vs inferred. The READMEs say which numbers came from a
	30	counter and which are explanations.