vole
Clone
Read only, no credentials:
https://rodent.software/git/vole.git
Write over SSH with a registered key:
git clone ssh://git@0.0.0.0:2222/vole.git
A free and open-source forge that lives in your Git repo.
Phase 5: an object is served only through a repository that holds it
65b88b3b1940623b6f7fbc686b19e986497c9ff2 · Keenan Nemetz · 16d ago · parent bf1e027c
Phase 5: an object is served only through a repository that holds it The store was instance-wide and content-addressed, so an object was fetchable through any repository whose read rule you passed, provided you knew its sha256. That is now impossible, and the fix is structural rather than a check: the pool still holds the bytes once, and each repository has a hard link into a namespace of its own (data/lfs/repos/<repo>/…). A fetch reads only that namespace, so "which repository owns these bytes" stops being a database question that can go stale and becomes a fact about the filesystem — and gc reconciles the links from the refs, which keeps the promise that derived state can always be thrown away and rebuilt. The alternatives were worse. A (repo, oid) table would have made the read path depend on the index, so deleting data/index.db — which the roadmap promises loses nothing — would have stopped LFS fetches until a rebuild. Namespacing the storage itself would have cost the deduplication the store exists for. Hard links give both: one copy of the bytes, one directory entry per repository, and removing one repository's link leaves the other's data alone. Three consequences that had to be handled rather than assumed: - pre-receive checks pointers against the pushing repository, not the pool. A pointer to an object that exists only in another repository would otherwise pass the push and then 404 on clone. The client re-uploads, the pool deduplicates and the link lands, which is the transfer cost isolation implies. - The sweep walks per repository, because reachability is per repository: an object reachable in one and not another keeps the first link and loses the second while the bytes stay. So `vole gc` is also the repair — it links what a repository reaches and unlinks what it no longer does. - An instance upgrading from the shared pool has no namespaces at all, so every fetch would 404 until somebody read an upgrade note. server.New notices a store with no namespace and runs the link pass in the background once, using gc.Link — the half of gc that creates links and deletes nothing, so an interrupted pass leaves the store more reachable than it found it. A repository rename moves its namespace with it. Reference counting stays rejected, and this makes the argument stronger: the thing that repairs the store is the same walk that cleans it, and it cannot invent a reference that does not exist. Measured: TestAnObjectIsInvisibleToARepositoryThatDoesNotHoldIt and the server level TestAnObjectIsNotFetchableThroughAnotherRepository (404 over the LFS endpoint for an object the shared store demonstrably has); TestTwoRepositoriesShareOneCopyOfTheBytes proves os.SameFile, so isolation costs no deduplication; TestOneRepositoryLosingAPointerKeepsTheOthersLink and TestLinkRepairsAMissingNamespace pin the per-repository sweep and the repair; TestAnUnscopedStoreIsLinkedOnStartup covers the upgrade path. Phase 5 is now complete: locking and object scoping both landed.
docs/roadmap.md +41−8
| @@ -608,7 +608,7 @@ a push whose spool file was never written still runs ✅ (the refs are reconcile | |||
| 608 | 608 | by `TestReconcileRebuildsAQueueThatWasDeleted`, which deletes the queue and its WAL and | |
| 609 | 609 | gets the live tip queued again). | |
| 610 | 610 | ||
| 611 | ### Phase 5 — LFS — ✅ complete (object scoping outstanding) | ||
| 611 | ### Phase 5 — LFS — ✅ complete | ||
| 612 | 612 | ||
| 613 | 613 | ✅ **Slice 1 — the store, and reads through the proxy.** Objects live at | |
| 614 | 614 | `data/lfs/objects/<aa>/<bb>/<sha256>`, shared by every repository on the instance, and | |
| @@ -652,13 +652,45 @@ Remaining: | |||
| 652 | 652 | - (Artifact uploads from workflows landed in Phase 7 as run artifacts: a job declares | |
| 653 | 653 | `artifacts:`, the runner publishes the matches to the object store, and the run page | |
| 654 | 654 | serves them. See Phase 7 slice 6.) | |
| 655 | - Per-repository object scoping. The store is instance-wide, so an object is fetchable | ||
| 656 | from any repository whose read rule you pass, if you know its id. Acceptable for | ||
| 657 | release artifacts, worth revisiting if repositories ever need to be isolated from | ||
| 658 | each other. | ||
| 659 | - Reference counting was replaced by mark-and-sweep. A counter that is one too low | ||
| 660 | deletes somebody's artifact after a crash; a walk over the refs cannot be wrong about | ||
| 661 | what is reachable, only slow — and `gc` is an operator command, not a hot path. | ||
| 655 | ✅ **Slice 6 — objects are scoped to the repository that owns them.** The store was | ||
| 656 | instance-wide and content-addressed, so an object was fetchable through *any* repository | ||
| 657 | whose read rule you passed, provided you knew its sha256. That is now impossible, and the | ||
| 658 | fix is structural rather than a check: the pool still holds the bytes once, and each | ||
| 659 | repository has a hard link into a namespace of its own (`data/lfs/repos/<repo>/…`). A fetch | ||
| 660 | reads only that namespace, so "which repository owns these bytes" stops being a database | ||
| 661 | question that can go stale and becomes a fact about the filesystem — and `gc` reconciles | ||
| 662 | the links from the refs, which keeps the project's promise that derived state can always | ||
| 663 | be thrown away and rebuilt. | ||
| 664 | |||
| 665 | The alternatives were worse. A `(repo, oid)` table would have made the read path depend on | ||
| 666 | the index, so deleting `data/index.db` — which the roadmap promises loses nothing — would | ||
| 667 | have stopped LFS fetches until a rebuild. Namespacing the storage itself would have cost | ||
| 668 | the deduplication the store exists for. Hard links give both: one copy of the bytes, one | ||
| 669 | directory entry per repository, and removing one repository's link leaves the other's data | ||
| 670 | untouched. | ||
| 671 | |||
| 672 | Three consequences that had to be handled, not assumed: | ||
| 673 | |||
| 674 | - **`pre-receive` checks pointers against the pushing repository**, not against the pool. A | ||
| 675 | pointer to an object that exists only in another repository would otherwise pass the push | ||
| 676 | and then 404 on clone. The client re-uploads, the pool deduplicates, and the link lands — | ||
| 677 | which is exactly the transfer cost isolation implies. | ||
| 678 | - **The sweep walks per repository**, because reachability is per repository: an object | ||
| 679 | reachable in one and not another keeps the first link and loses the second, while the | ||
| 680 | bytes stay. `vole gc` is therefore also the repair — it links what a repository reaches | ||
| 681 | and unlinks what it no longer does. | ||
| 682 | - **An instance upgrading from the shared pool has no namespaces at all.** Every fetch | ||
| 683 | would 404 until somebody read an upgrade note. So `server.New` notices a store with no | ||
| 684 | namespace and runs the link pass in the background once, using `gc.Link` — the half of gc | ||
| 685 | that creates links and deletes nothing, so an interrupted pass leaves the store more | ||
| 686 | reachable than it found it and the next boot finishes the job. A repository rename moves | ||
| 687 | its namespace with it. | ||
| 688 | |||
| 689 | Reference counting stays rejected. A counter that is one too low deletes somebody's | ||
| 690 | artifact after a crash; a walk over the refs cannot be wrong about what is reachable, only | ||
| 691 | slow — and `gc` is an operator command, not a hot path. The link pass makes that argument | ||
| 692 | stronger: the thing that repairs the store is the same walk that cleans it, and it cannot | ||
| 693 | invent a reference that does not exist. | ||
| 662 | 694 | ||
| 663 | 695 | ✅ **Slice 5 — file locking.** `git lfs lock <path>` works over the SSH transfer, so a | |
| 664 | 696 | binary that git cannot merge can be reserved by one person before they edit it: a push | |
| @@ -1139,6 +1171,7 @@ Filled in as phases land. Append; don't rewrite. | |||
| 1139 | 1171 | | 2026-10-23 | A runner's `--repos` filter is applied by the server, as an intersection with the allowlist | A filter and a grant are different things: `--repos` says what one machine is *for*, and the runner allowlist says what its account *may do*. Applying the filter server-side, against the grant, means the client's list can only shrink the answer — if the server honoured the request list directly, an SSH flag would have become an access-control decision, and the first person to notice would be whoever read the flag's name and assumed otherwise. An empty filter means "no narrowing" rather than "nothing" so an unfiltered runner keeps following allowlist edits instead of freezing at connect time. The cost is a filter that matches nothing, which presents as a healthy idle runner, so `ping` reports the grant and the runner refuses at startup instead. `Lease` consequently takes a struct: it now holds two lists of names, and adjacent `[]string` arguments are the shape that transposes silently | | |
| 1140 | 1172 | | 2026-10-23 | `runs-on` labels are pinned at enqueue, and a runner's labels are a claim rather than a grant | Pinning reuses the rule that already governs steps and artifact patterns: a workflow edited after a push must not change what an already-queued job is. That settles the derived-store question too — `run_labels` is queue state, not history, because a finished run's requirement is recoverable from the pinned workflow blob, so `index rebuild` does not restore labels and nothing needs it to. The asymmetry with `--repos` is the interesting part: a runner's labels are a claim about itself, so lying only wins it work it will do badly, while a runner's repository list is enforced by the server because an SSH flag must not become an access-control decision. Both are in the same lease request and neither is trusted for the same reason. Labels are lowercase and validated by `pre-receive` and by the runner with one shared rule, because a label no runner can carry leaves a job queued forever behind an error nobody sees | | |
| 1141 | 1173 | | 2026-10-23 | LFS locking rides the SSH transfer protocol, and the refusal lives in `pre-receive` | The HTTP locking API is a write surface, and this instance has none: HTTP is anonymous read-only. The SSH adapter defines `lock`, `list-lock` and `unlock`, so locking costs no new authentication — the key that pushed is the key that locks. Where the refusal lives is the part worth remembering. git-lfs checks locks on the *client*, which means a plain `git push`, or a client with its hook disabled, walks past a lock entirely — so the rule also sits in `pre-receive`, at the one place this forge enforces everything else. The client-side check becomes a faster error message rather than the authority. Narrow on purpose: only the paths the push writes, so branching and merging untouched history are unaffected. Admin override is audited, for the same reason force-push is: a lock whose holder has left would otherwise make a path unpushable forever | | |
| 1174 | | 2026-10-23 | An LFS object is served only through a repository that holds a link to it | The store is content-addressed, so a plain lookup by id let anybody who knew a sha256 read another repository's artifact through a repository whose read rule they passed. Two fixes were available and one was rejected: an oid→repo table would have made the read path depend on derived state, so deleting index.db — which the project promises loses nothing — would have broken every LFS fetch until a rebuild. Hard links into a per-repository namespace keep the promise (the links are reconciled from the refs, so they are rebuildable too) and keep the deduplication (one copy of the bytes, two directory entries). The consequences are the interesting part: pre-receive has to check pointers against the *pushing* repository or a cross-repo pointer passes the push and 404s on clone; the sweep has to walk per repository, which makes it the repair tool as well; and an instance upgrading from the shared pool has no namespaces, so the server runs the link pass on first boot rather than leaving every fetch broken behind an upgrade note | | ||
| 1142 | 1175 | ||
| 1143 | 1176 | ## Open questions | |
| 1144 | 1177 | ||