Sign in

Search issue titles and bodies, change requests and comments. Private repositories you cannot read are not included.

vole
Clone

Read only, no credentials:

https://rodent.software/vole.git

Write over SSH with a registered key:

git clone ssh://git@rodent.software:2222/vole.git

A free and open-source forge that lives in your Git repo.

Phase 5: an object is served only through a repository that holds it

65b88b3b1940623b6f7fbc686b19e986497c9ff2 · Keenan Nemetz · 17d ago · parent bf1e027c

Phase 5: an object is served only through a repository that holds it

The store was instance-wide and content-addressed, so an object was fetchable
through any repository whose read rule you passed, provided you knew its
sha256. That is now impossible, and the fix is structural rather than a check:
the pool still holds the bytes once, and each repository has a hard link into
a namespace of its own (data/lfs/repos/<repo>/…). A fetch reads only that
namespace, so "which repository owns these bytes" stops being a database
question that can go stale and becomes a fact about the filesystem — and gc
reconciles the links from the refs, which keeps the promise that derived state
can always be thrown away and rebuilt.

The alternatives were worse. A (repo, oid) table would have made the read path
depend on the index, so deleting data/index.db — which the roadmap promises
loses nothing — would have stopped LFS fetches until a rebuild. Namespacing
the storage itself would have cost the deduplication the store exists for.
Hard links give both: one copy of the bytes, one directory entry per
repository, and removing one repository's link leaves the other's data alone.

Three consequences that had to be handled rather than assumed:

- pre-receive checks pointers against the pushing repository, not the pool. A
  pointer to an object that exists only in another repository would otherwise
  pass the push and then 404 on clone. The client re-uploads, the pool
  deduplicates and the link lands, which is the transfer cost isolation
  implies.
- The sweep walks per repository, because reachability is per repository: an
  object reachable in one and not another keeps the first link and loses the
  second while the bytes stay. So `vole gc` is also the repair — it links what
  a repository reaches and unlinks what it no longer does.
- An instance upgrading from the shared pool has no namespaces at all, so
  every fetch would 404 until somebody read an upgrade note. server.New
  notices a store with no namespace and runs the link pass in the background
  once, using gc.Link — the half of gc that creates links and deletes nothing,
  so an interrupted pass leaves the store more reachable than it found it.
  A repository rename moves its namespace with it.

Reference counting stays rejected, and this makes the argument stronger: the
thing that repairs the store is the same walk that cleans it, and it cannot
invent a reference that does not exist.

Measured: TestAnObjectIsInvisibleToARepositoryThatDoesNotHoldIt and the server
level TestAnObjectIsNotFetchableThroughAnotherRepository (404 over the LFS
endpoint for an object the shared store demonstrably has);
TestTwoRepositoriesShareOneCopyOfTheBytes proves os.SameFile, so isolation
costs no deduplication; TestOneRepositoryLosingAPointerKeepsTheOthersLink and
TestLinkRepairsAMissingNamespace pin the per-repository sweep and the repair;
TestAnUnscopedStoreIsLinkedOnStartup covers the upgrade path.

Phase 5 is now complete: locking and object scoping both landed.

internal/lfs/lfs_test.go +13−13

@@ -61,14 +61,14 @@ func TestStorePutOpenDedupe(t *testing.T) {
61 61 data := []byte("some bytes worth storing")
62 62 oid := oidOf(data)
63 63
64 n, created, err := s.Put(oid, int64(len(data)), bytes.NewReader(data))
64 n, created, err := s.Put("demo", oid, int64(len(data)), bytes.NewReader(data))
65 65 if err != nil || n != int64(len(data)) || !created {
66 66 t.Fatalf("Put = %d, %v, %v; want %d, true, nil", n, created, err, len(data))
67 67 }
68 if size, err := s.Stat(oid); err != nil || size != int64(len(data)) {
68 if size, err := s.Stat("demo", oid); err != nil || size != int64(len(data)) {
69 69 t.Fatalf("Stat = %d, %v", size, err)
70 70 }
71 f, size, err := s.Open(oid)
71 f, size, err := s.Open("demo", oid)
72 72 if err != nil {
73 73 t.Fatal(err)
74 74 }
@@ -78,7 +78,7 @@ func TestStorePutOpenDedupe(t *testing.T) {
78 78 t.Fatalf("Open returned %q (%d bytes)", got, size)
79 79 }
80 80 // The same content twice is one object.
81 if _, created, err := s.Put(oid, int64(len(data)), bytes.NewReader(data)); err != nil || created {
81 if _, created, err := s.Put("demo", oid, int64(len(data)), bytes.NewReader(data)); err != nil || created {
82 82 t.Fatalf("second Put created=%v err=%v; want deduplicated", created, err)
83 83 }
84 84
@@ -89,7 +89,7 @@ func TestStorePutOpenDedupe(t *testing.T) {
89 89 if err := s.Remove(oid); err != nil {
90 90 t.Fatal(err)
91 91 }
92 if s.Has(oid) {
92 if s.Has("demo", oid) {
93 93 t.Fatal("object still present after Remove")
94 94 }
95 95 if err := s.Remove(oid); err != nil {
@@ -103,28 +103,28 @@ func TestStoreRefusesWhatDoesNotMatchItsKey(t *testing.T) {
103 103 data := []byte("real content")
104 104 oid := oidOf([]byte("different content"))
105 105
106 if _, _, err := s.Put(oid, int64(len(data)), bytes.NewReader(data)); !errors.Is(err, ErrMismatch) {
106 if _, _, err := s.Put("demo", oid, int64(len(data)), bytes.NewReader(data)); !errors.Is(err, ErrMismatch) {
107 107 t.Fatalf("Put of mismatched content = %v; want ErrMismatch", err)
108 108 }
109 if s.Has(oid) {
109 if s.Has("demo", oid) {
110 110 t.Fatal("mismatched content was stored under the wrong id")
111 111 }
112 112
113 113 tight := New(t.TempDir(), 1024)
114 114 big := bytes.Repeat([]byte("x"), 2048)
115 if _, _, err := tight.Put(oidOf(big), 0, bytes.NewReader(big)); !errors.Is(err, ErrTooLarge) {
115 if _, _, err := tight.Put("demo", oidOf(big), 0, bytes.NewReader(big)); !errors.Is(err, ErrTooLarge) {
116 116 t.Fatalf("oversized Put = %v; want ErrTooLarge", err)
117 117 }
118 118
119 119 small := []byte("declared size lies")
120 if _, _, err := s.Put(oidOf(small), 9999, bytes.NewReader(small)); !errors.Is(err, ErrMismatch) {
120 if _, _, err := s.Put("demo", oidOf(small), 9999, bytes.NewReader(small)); !errors.Is(err, ErrMismatch) {
121 121 t.Fatalf("Put with a lying size = %v; want ErrMismatch", err)
122 122 }
123 123
124 if _, err := s.Stat("not-an-oid"); !errors.Is(err, ErrBadOID) {
124 if _, err := s.Stat("demo", "not-an-oid"); !errors.Is(err, ErrBadOID) {
125 125 t.Fatalf("Stat of a malformed id = %v", err)
126 126 }
127 if _, err := s.Stat(strings.Repeat("f", 64)); !errors.Is(err, ErrNotFound) {
127 if _, err := s.Stat("demo", strings.Repeat("f", 64)); !errors.Is(err, ErrNotFound) {
128 128 t.Fatalf("Stat of an absent object = %v", err)
129 129 }
130 130 }
@@ -137,7 +137,7 @@ func TestStoreWalkFindsShardedObjects(t *testing.T) {
137 137 for i := 0; i < 3; i++ {
138 138 data := []byte(strings.Repeat("ab", i+1))
139 139 oid := oidOf(data)
140 if _, _, err := s.Put(oid, int64(len(data)), bytes.NewReader(data)); err != nil {
140 if _, _, err := s.Put("demo", oid, int64(len(data)), bytes.NewReader(data)); err != nil {
141 141 t.Fatal(err)
142 142 }
143 143 want = append(want, oid)
@@ -153,7 +153,7 @@ func TestStoreWalkFindsShardedObjects(t *testing.T) {
153 153 t.Fatalf("Walk saw %d objects, want %d (%v)", len(seen), len(want), seen)
154 154 }
155 155 for _, oid := range want {
156 if !s.Has(oid) {
156 if !s.Has("demo", oid) {
157 157 t.Fatalf("Walk did not see %s, which is at %s", oid, s.ObjectPath(oid))
158 158 }
159 159 }