Sign in

Search issue titles and bodies, change requests and comments. Private repositories you cannot read are not included.

vole
Clone

Read only, no credentials:

https://rodent.software/vole.git

Write over SSH with a registered key:

git clone ssh://git@rodent.software:2222/vole.git

A free and open-source forge that lives in your Git repo.

Phase 5: LFS objects, over SSH for writes and the proxy for reads

64ac5071a173e4e9ce8bc8d7170f9b0411bee52a · Keenan Nemetz · 18d ago · parent 35eaeccf

Phase 5: LFS objects, over SSH for writes and the proxy for reads

Large files leave the git object database and land in one content-addressed
store per instance. Reads are anonymous like clone; writes ride the SSH key
that already authenticates the push, so no token exists anywhere.

- internal/lfs: sha256-addressed store that verifies content and size before
  an object becomes visible; parser for the pointer format a client writes
- HTTP: POST .../info/lfs/objects/batch for downloads, GET .../objects/<oid>
  with the object id as ETag; upload over HTTP is refused and names SSH
- SSH: git-lfs-transfer speaking the pure SSH protocol (pkt-line), so
  `git lfs push` needs nothing but the key at the handshake
- pre-receive: a pointer whose object is missing, or whose size disagrees
  with the stored object, fails the push that introduced it; a binary over
  lfs_warn_bytes is accepted with a `git lfs track` suggestion. Checks run
  before the protected-branch gate, because an admin's approval bypass is
  about review, not about whether the commit can be checked out
- runs: a finished log becomes a content-addressed object in the same write
  that records the status, served at /r/<repo>/runs/<id>/log
- vole gc [--dry-run] [--grace]: marks pointer targets from every ref in
  every repository plus archived logs, sweeps the rest. Mark-and-sweep
  rather than reference counts: a counter that is one too low deletes an
  artifact, a walk over the refs cannot be wrong about what is reachable

internal/hook/hook.go +30−8

@@ -15,6 +15,7 @@ import (
15 15 "github.com/rodent-software/vole/internal/gitx"
16 16 "github.com/rodent-software/vole/internal/identity"
17 17 "github.com/rodent-software/vole/internal/issues"
18 "github.com/rodent-software/vole/internal/lfs"
18 19 "github.com/rodent-software/vole/internal/repo"
19 20 "github.com/rodent-software/vole/internal/requests"
20 21 "github.com/rodent-software/vole/internal/runs"
@@ -36,6 +37,12 @@ type Config struct {
36 37 // RunsDB is the run queue, read here so required checks are enforced at the
37 38 // same single point as everything else.
38 39 RunsDB string
40 // LFS is the object store, read here so a pointer that names an object the
41 // server does not have fails the push that introduced it.
42 LFS *lfs.Store
43 // LFSWarnBytes flags binaries committed straight into git above this size.
44 // Zero disables the warning.
45 LFSWarnBytes int64
39 46 // AuditPath receives lines about privileged metadata operations. Empty means
40 47 // no audit sink, which is only acceptable for tests.
41 48 AuditPath string
@@ -159,6 +166,29 @@ func PreReceive(cfg Config) int {
159 166 }
160 167 }
161 168
169 // Integrity checks run before the protected-branch gate, because an admin's
170 // approval bypass is about review, not about whether the repository works:
171 // a pointer with no object behind it is broken no matter who pushed it.
172 if cfg.MaxBlobBytes > 0 && c.new != gitx.ZeroSHA {
173 overs, err := oversizedBlobs(g, c, cfg.MaxBlobBytes)
174 if err != nil {
175 problems = append(problems, fmt.Sprintf("%s: cannot check object sizes: %v", c.ref, err))
176 continue
177 }
178 problems = append(problems, overs...)
179 }
180 // LFS objects live outside the pack, so receive-pack cannot notice that a
181 // pointer names nothing. This is the only place that can.
182 if cfg.LFS != nil && c.new != gitx.ZeroSHA {
183 lfsProblems, warnings, err := lfsFindings(g, c.new, cfg.LFS, cfg.LFSWarnBytes)
184 if err != nil {
185 problems = append(problems, fmt.Sprintf("%s: cannot check LFS pointers: %v", c.ref, err))
186 } else {
187 problems = append(problems, lfsProblems...)
188 printWarnings(cfg.Out, warnings)
189 }
190 }
191
162 192 if a.IsProtected(c.ref) {
163 193 // Admins keep the bypass they always had. Everyone else needs an
164 194 // approved request that names the exact commit they are landing.
@@ -229,14 +259,6 @@ func PreReceive(cfg Config) int {
229 259 problems = append(problems, prob)
230 260 }
231 261 }
232 if cfg.MaxBlobBytes > 0 && c.new != gitx.ZeroSHA {
233 overs, err := oversizedBlobs(g, c, cfg.MaxBlobBytes)
234 if err != nil {
235 problems = append(problems, fmt.Sprintf("%s: cannot check object sizes: %v", c.ref, err))
236 continue
237 }
238 problems = append(problems, overs...)
239 }
240 262 }
241 263
242 264 if len(problems) == 0 {