P1: Every metadata write rewrites the whole vole tree

Why

Filing an issue from the web takes about 12 seconds on a tree the size of this repository’s vole branch, and it gets slower every time anything is written, including CI. The cause is that git.CommitFiles (core/git/trees.go) implements a one-file change as a full tree rewrite:

• Repo.FlatTree spawns one git cat-file blob process per blob.
• WriteFlatTree spawns one git hash-object -w per blob and one git mktree per directory, including every unchanged directory.

Every metadata writer goes through it: metadata.CreateIssue, comments, edits, approvals, requests, releases, and ci/runs.WriteRecord.

Measurements

A throwaway bare repo on this host, seeded to match the current branch (959 files on vole):

That is about 13 ms per existing file, per write. Within one create, roughly 3.3 s reads the tree and 8.6 s rewrites it. The same 959 blobs through one git cat-file --batch call take 55 ms, so the cost is process spawning, not git.

The Phase 0 spike measured the same seam from the other direction: one shared ref managed 30 writes/s with 22 CAS retries per write at 2,000 entities, while sharded refs managed 1,240/s naive and 6,966/s batched (c5626f1, docs/spikes/ref-cas/README.md in that commit). Making the tree walk cheap does not remove single-ref contention, but it makes each attempt short enough that contention stops dominating.

Scope

• Read the tree in one git cat-file --batch call instead of one process per blob.
• Make the write proportional to what changed: reuse the old blob and subtree ids for untouched paths, and rewrite only the trees on the path to the root.
• Keep CommitFiles the single write path, with the CAS and retry behavior it documents.
• Account for the CAS replay cost: a lost race currently re-runs the whole mutate/hash/write. Either serialize local writers per repository, or make the retry cheap enough that it does not matter.
• Add a regression test that creates an issue in a repo with many unrelated files and asserts the git work does not scale with the tree (or a benchmark with a bound).

Acceptance

• Filing an issue on a 2,000-file vole branch is under 200 ms, and the time does not grow with unrelated files.
• A comment on an issue with a long history costs the same as the first comment.
• go test ./... passes, including the metadata, limits, and run-record tests.
• A benchmark in core/git or metadata shows the flat cost, so the next regression is visible.

Related

• Machine-written run records are 849 of those 959 files and grow about 7 per CI run: see #80.
• The full index rebuild after a write is #81.
• Phase 0 measurement: commit c5626f1.