vole

Clone

Read only, no credentials:

https://rodent.software/vole.git

Write over SSH with a registered key:

git clone ssh://git@rodent.software:2222/vole.git

A free and open-source forge that lives in your Git repo.

Phase 5: LFS objects, over SSH for writes and the proxy for reads

64ac5071a173e4e9ce8bc8d7170f9b0411bee52a · Keenan Nemetz · 18d ago · parent 35eaeccf

Phase 5: LFS objects, over SSH for writes and the proxy for reads

Large files leave the git object database and land in one content-addressed
store per instance. Reads are anonymous like clone; writes ride the SSH key
that already authenticates the push, so no token exists anywhere.

- internal/lfs: sha256-addressed store that verifies content and size before
  an object becomes visible; parser for the pointer format a client writes
- HTTP: POST .../info/lfs/objects/batch for downloads, GET .../objects/<oid>
  with the object id as ETag; upload over HTTP is refused and names SSH
- SSH: git-lfs-transfer speaking the pure SSH protocol (pkt-line), so
  `git lfs push` needs nothing but the key at the handshake
- pre-receive: a pointer whose object is missing, or whose size disagrees
  with the stored object, fails the push that introduced it; a binary over
  lfs_warn_bytes is accepted with a `git lfs track` suggestion. Checks run
  before the protected-branch gate, because an admin's approval bypass is
  about review, not about whether the commit can be checked out
- runs: a finished log becomes a content-addressed object in the same write
  that records the status, served at /r/<repo>/runs/<id>/log
- vole gc [--dry-run] [--grace]: marks pointer targets from every ref in
  every repository plus archived logs, sweeps the rest. Mark-and-sweep
  rather than reference counts: a counter that is one too low deletes an
  artifact, a walk over the refs cannot be wrong about what is reachable

internal/config/config.go +20

@@ -21,6 +21,11 @@ type Config struct {
	21	21	// ReceiveMaxBytes caps a whole push, enforced by git's own
	22	22	// receive.maxInputSize in each repo rather than by buffering in the proxy.
	23	23	ReceiveMaxBytes int64 `toml:"receive_max_bytes"`
		24	// LFSObjectMaxBytes caps a single LFS object.
		25	LFSObjectMaxBytes int64 `toml:"lfs_object_max_bytes"`
		26	// LFSWarnBytes makes a push that commits a binary larger than this carry a
		27	// suggestion to track it with git-lfs. Zero disables the warning.
		28	LFSWarnBytes int64 `toml:"lfs_warn_bytes"`
	24	29	// LeaseSeconds is how long a runner may hold a job without a heartbeat before
	25	30	// the job is handed to somebody else.
	26	31	LeaseSeconds int `toml:"lease_seconds"`
@@ -35,6 +40,9 @@ func defaults() *Config {
	35	40	SSHAddr: "127.0.0.1:2222",
	36	41	MaxBlobBytes: 100 << 20, // 100 MiB per blob
	37	42	ReceiveMaxBytes: 1 << 30, // 1 GiB per push
		43
		44	LFSObjectMaxBytes: 2 << 30, // 2 GiB per LFS object
		45	LFSWarnBytes: 5 << 20, // flag binaries over 5 MiB that are not in LFS
	38	46	}
	39	47	}
	40	48
@@ -72,6 +80,10 @@ func (c *Config) RunsDB() string { return filepath.Join(c.DataDir, "runs.db") }
	72	80	// SpoolDir receives accepted push events from post-receive hooks.
	73	81	func (c *Config) SpoolDir() string { return filepath.Join(c.DataDir, "queue", "spool") }
	74	82
		83	// LFSDir is the instance-wide content-addressed store. Objects are shared by
		84	// every repository: the same artifact pushed twice costs one copy.
		85	func (c *Config) LFSDir() string { return filepath.Join(c.DataDir, "lfs") }
		86
	75	87	// Lease returns how long a runner may hold a job without a heartbeat.
	76	88	func (c *Config) Lease() time.Duration {
	77	89	if c.LeaseSeconds <= 0 {
@@ -97,6 +109,14 @@ ssh_addr = "127.0.0.1:2222"
	97	109	# git-lfs instead.
	98	110	max_blob_bytes = 104857600
	99	111
		112	# LFS objects live in a content-addressed store shared by every repository.
		113	# This caps one object; anything larger is refused rather than spooled.
		114	lfs_object_max_bytes = 2147483648
		115
		116	# A push that commits a binary bigger than this, and is not already an LFS
		117	# pointer, gets a warning naming the git lfs track command. Zero disables it.
		118	lfs_warn_bytes = 5242880
		119
	100	120	# Cap a whole push. Enforced by git's receive.maxInputSize in each repo, because
	101	121	# receive-pack indexes a pack in memory and a large push costs its own size in
	102	122	# RSS.