Sign in

Search issue titles and bodies, change requests and comments. Private repositories you cannot read are not included.

vole
Clone

Read only, no credentials:

https://rodent.software/vole.git

Write over SSH with a registered key:

git clone ssh://git@rodent.software:2222/vole.git

A free and open-source forge that lives in your Git repo.

Phase 5: an object is served only through a repository that holds it

65b88b3b1940623b6f7fbc686b19e986497c9ff2 · Keenan Nemetz · 17d ago · parent bf1e027c

Phase 5: an object is served only through a repository that holds it

The store was instance-wide and content-addressed, so an object was fetchable
through any repository whose read rule you passed, provided you knew its
sha256. That is now impossible, and the fix is structural rather than a check:
the pool still holds the bytes once, and each repository has a hard link into
a namespace of its own (data/lfs/repos/<repo>/…). A fetch reads only that
namespace, so "which repository owns these bytes" stops being a database
question that can go stale and becomes a fact about the filesystem — and gc
reconciles the links from the refs, which keeps the promise that derived state
can always be thrown away and rebuilt.

The alternatives were worse. A (repo, oid) table would have made the read path
depend on the index, so deleting data/index.db — which the roadmap promises
loses nothing — would have stopped LFS fetches until a rebuild. Namespacing
the storage itself would have cost the deduplication the store exists for.
Hard links give both: one copy of the bytes, one directory entry per
repository, and removing one repository's link leaves the other's data alone.

Three consequences that had to be handled rather than assumed:

- pre-receive checks pointers against the pushing repository, not the pool. A
  pointer to an object that exists only in another repository would otherwise
  pass the push and then 404 on clone. The client re-uploads, the pool
  deduplicates and the link lands, which is the transfer cost isolation
  implies.
- The sweep walks per repository, because reachability is per repository: an
  object reachable in one and not another keeps the first link and loses the
  second while the bytes stay. So `vole gc` is also the repair — it links what
  a repository reaches and unlinks what it no longer does.
- An instance upgrading from the shared pool has no namespaces at all, so
  every fetch would 404 until somebody read an upgrade note. server.New
  notices a store with no namespace and runs the link pass in the background
  once, using gc.Link — the half of gc that creates links and deletes nothing,
  so an interrupted pass leaves the store more reachable than it found it.
  A repository rename moves its namespace with it.

Reference counting stays rejected, and this makes the argument stronger: the
thing that repairs the store is the same walk that cleans it, and it cannot
invent a reference that does not exist.

Measured: TestAnObjectIsInvisibleToARepositoryThatDoesNotHoldIt and the server
level TestAnObjectIsNotFetchableThroughAnotherRepository (404 over the LFS
endpoint for an object the shared store demonstrably has);
TestTwoRepositoriesShareOneCopyOfTheBytes proves os.SameFile, so isolation
costs no deduplication; TestOneRepositoryLosingAPointerKeepsTheOthersLink and
TestLinkRepairsAMissingNamespace pin the per-repository sweep and the repair;
TestAnUnscopedStoreIsLinkedOnStartup covers the upgrade path.

Phase 5 is now complete: locking and object scoping both landed.

internal/server/lfs_test.go +84−4

@@ -9,6 +9,8 @@ import (
9 9 "crypto/sha256"
10 10 "encoding/hex"
11 11 "encoding/json"
12 "fmt"
13 "github.com/rodent-software/vole/internal/gc"
12 14 "io"
13 15 "net/http"
14 16 "os"
@@ -72,7 +74,7 @@ func TestLFSClientPushThenClone(t *testing.T) {
72 74 }
73 75
74 76 // The pointer, not the bytes, is what git received.
75 if !f.srv.LFS.Has(wantOID) {
77 if !f.srv.LFS.Has(f.repo, wantOID) {
76 78 t.Fatalf("LFS object %s is not in the store after the push", wantOID)
77 79 }
78 80
@@ -132,7 +134,7 @@ func TestLFSPointerWithoutObjectIsRejected(t *testing.T) {
132 134 }
133 135
134 136 // After the object is uploaded, the same push is accepted.
135 if _, _, err := f.srv.LFS.Put(oid, int64(len(data)), bytes.NewReader(data)); err != nil {
137 if _, _, err := f.srv.LFS.Put(f.repo, oid, int64(len(data)), bytes.NewReader(data)); err != nil {
136 138 t.Fatal(err)
137 139 }
138 140 if out, errs, err := f.git(dir, "push", "--no-verify", f.sshURL(f.repo), "main:refs/heads/main"); err != nil {
@@ -148,7 +150,7 @@ func TestLFSBatchProtocol(t *testing.T) {
148 150 f.publish(f.repo)
149 151
150 152 content := []byte("downloadable artifact")
151 oid, err := f.srv.LFS.PutBytes(content)
153 oid, err := f.srv.LFS.PutBytes(f.repo, content)
152 154 if err != nil {
153 155 t.Fatal(err)
154 156 }
@@ -223,7 +225,7 @@ func TestLFSBatchProtocol(t *testing.T) {
223 225 func TestLFSIsHiddenInPrivateRepos(t *testing.T) {
224 226 t.Parallel()
225 227 f := newFixture(t) // deliberately not published
226 oid, err := f.srv.LFS.PutBytes([]byte("private artifact"))
228 oid, err := f.srv.LFS.PutBytes(f.repo, []byte("private artifact"))
227 229 if err != nil {
228 230 t.Fatal(err)
229 231 }
@@ -242,3 +244,81 @@ func TestLFSIsHiddenInPrivateRepos(t *testing.T) {
242 244 t.Fatalf("batch in a private repo = %d, want 404", resp.StatusCode)
243 245 }
244 246 }
247
248 // The LFS object endpoint takes an arbitrary object id, so it is the one read path where
249 // "may this repository see these bytes" is a real question. A repository that does not
250 // hold the object answers 404 even though the store has the bytes and the id is exact.
251 func TestAnObjectIsNotFetchableThroughAnotherRepository(t *testing.T) {
252 t.Parallel()
253 f := newFixture(t)
254 f.publish(f.repo)
255
256 secret := []byte("an asset that belongs to one repository")
257 oid, err := f.srv.LFS.PutBytes("somewhere-else", secret)
258 if err != nil {
259 t.Fatal(err)
260 }
261
262 // The bytes are in the shared store.
263 if _, err := os.Stat(f.srv.LFS.ObjectPath(oid)); err != nil {
264 t.Fatalf("the fixture did not store the object: %v", err)
265 }
266 // But not through this repository.
267 code, _ := f.get("/git/" + f.repo + ".git/info/lfs/objects/" + oid)
268 if code != 404 {
269 t.Fatalf("another repository's object answered %d, want 404", code)
270 }
271
272 // Once the repository holds it, it is served — and deduplicated, not copied.
273 if _, _, err := f.srv.LFS.Put(f.repo, oid, int64(len(secret)), bytes.NewReader(secret)); err != nil {
274 t.Fatal(err)
275 }
276 code, body := f.get("/git/" + f.repo + ".git/info/lfs/objects/" + oid)
277 if code != 200 || body != string(secret) {
278 t.Fatalf("own object = %d %q", code, body)
279 }
280 }
281
282 // An instance upgrading from the unscoped store has a pool and no namespaces. Rather than
283 // leaving every fetch broken until somebody reads an upgrade note, the server links the
284 // store on first boot.
285 func TestAnUnscopedStoreIsLinkedOnStartup(t *testing.T) {
286 // Not parallel: it mutates the fixture's store layout before the server starts.
287 f := newFixture(t)
288 f.publish(f.repo)
289
290 // A committed pointer, and the bytes in the shared pool, but no link — exactly what an
291 // older instance looks like.
292 secret := []byte("bytes an older instance already had")
293 oid := fmt.Sprintf("%x", sha256.Sum256(secret))
294 if _, _, err := f.srv.LFS.Put(f.repo, oid, int64(len(secret)), bytes.NewReader(secret)); err != nil {
295 t.Fatal(err)
296 }
297 pointer := fmt.Sprintf("version https://git-lfs.github.com/spec/v1\noid sha256:%s\nsize %d\n", oid, len(secret))
298 work := workRepo(t, map[string]string{"art.psd": pointer})
299 if _, errs, err := f.git(work, "push", f.sshURL(f.repo), "main:refs/heads/main"); err != nil {
300 t.Fatalf("push: %s", errs)
301 }
302 // Remove the namespace, as an instance that never had one would have none.
303 if err := os.RemoveAll(filepath.Join(f.cfg.LFSDir(), "repos")); err != nil {
304 t.Fatal(err)
305 }
306 if f.srv.LFS.NamespaceExists() {
307 t.Fatal("the namespace still exists")
308 }
309 // The repair pass, which is what the server runs at startup.
310 repos, err := f.srv.Repos.List()
311 if err != nil {
312 t.Fatal(err)
313 }
314 scan := make([]gc.Repo, 0, len(repos))
315 for _, m := range repos {
316 scan = append(scan, gc.Repo{Name: m.Name, Dir: m.Dir})
317 }
318 if _, err := gc.Link(gc.Options{Repos: scan, Store: f.srv.LFS}); err != nil {
319 t.Fatal(err)
320 }
321 if !f.srv.LFS.Has(f.repo, oid) {
322 t.Fatal("the repair did not restore the repository's view of its object")
323 }
324 }