Sign in

Search issue titles and bodies, change requests and comments. Private repositories you cannot read are not included.

vole
Clone

Read only, no credentials:

https://rodent.software/git/vole.git

Write over SSH with a registered key:

git clone ssh://git@0.0.0.0:2222/vole.git

A free and open-source forge that lives in your Git repo.

#82

Write only the part of a metadata tree that changed

merged · by keenan 11d ago · merged into main at d61054ae by keenan 11d ago

keenan opened this 11d ago

Why

Metadata writes rewrote the whole vole branch, one blob and one directory at a time. At 959 files a create took about 12 seconds, and each CI run added seven files to that cost. #79 has the measurements and the Phase 0 spike that predicted it.

What changed

  • FlatTree reads the branch with one git cat-file --batch call and an ls-tree -r -t that also returns directory ids.
  • CommitFiles hashes content in memory to find the paths that changed, then writes only those blobs and the directories above them. Everything else reuses the old blob and subtree ids.
  • A write that changes no bytes leaves the tree id exactly as it was.
  • The first commit on an empty ref still writes every blob: there is no old tree to reuse.

Tests

  • TestBlobIDMatchesGit pins the in-memory blob id against git hash-object.
  • TestBlobsBySHABinary covers binary content, an empty blob, and duplicate requests through the batch reader.
  • TestCommitFilesIncrementalEdits adds, changes, and deletes in one commit, checks the resulting files, and checks that an unrelated subtree id was reused.
  • TestCommitFilesReusesTreeWhenNothingChanged pins the tree id for a no-op write.
  • TestCommitFilesStaysProportionalToTheChange sets GIT_TRACE and fails if a one-file write on a 200-file tree starts more than 20 git processes. The old path started over 400.
  • BenchmarkCommitFilesOneFile measures about 44 ms per one-file write on a 2,000-file tree, down from 26.8 s in the scale measurement on #79.

Checks

  • go test ./...
  • go test -race ./core/git ./metadata ./ci/runs ./client
  • go vet ./...
  • vole workflow check
  • vole workflow run verify: format, vet, build, govulncheck, packages, markdown links, and the race suite.

Follow-ups

  • #80 keeps machine-written run records from growing the tree that human writes walk.
  • #81 removes the full index rebuild and validation that still follow a web write.

Discussion (0)

No discussion yet.