Sign in

Search issue titles and bodies, change requests and comments. Private repositories you cannot read are not included.

vole
Clone

Read only, no credentials:

https://rodent.software/vole.git

Write over SSH with a registered key:

git clone ssh://git@rodent.software:2222/vole.git

A free and open-source forge that lives in your Git repo.

Phase 7 slice 6: a job declares its outputs, the runner publishes them

213f4d6dc6707cc9199f97acdacd617740bfb84d · Keenan Nemetz · 17d ago · parent 4c0f2aef

Phase 7 slice 6: a job declares its outputs, the runner publishes them

Workflow artifacts are a declaration, not a CLI call in a step. A workflow is
validated at the push that introduces it, so a bad pattern fails the push; a
shell command calling vole would fail at run time, need the binary on a runner's
PATH, and get idempotency wrong under retries.

- workflow: artifacts: [{path: dist/*}] on a job, validated at push — relative,
  no '..', no '**', no trailing slash, a glob that parses. Patterns are pinned
  into the queued run beside the pinned steps
- runner: expands the patterns in its checkout after the steps pass (one segment
  at a time, .git excluded), uploads through the same SSH object transfer git-lfs
  uses, and reports the list with its finish. A pattern matching nothing fails the
  job: a silent empty artifact is the failure found a week later
- server: the artifact list rides the finish message, so a run's outputs and its
  result are one fact; nothing is recorded unless the lease is still held, and
  every oid must be in the store at the stated size, or the finish is refused
- storage: run_artifacts rows point at objects in the existing store. A build
  output is machine-written state, so it does not touch the metadata branch; the
  queue is its only index, and gc marks it from there the way it marks logs
- web: a run page (status, job, sha, log, artifacts) and
  /r/<repo>/runs/<id>/artifacts/<name> served from the store, ETag = oid, always
  an attachment, nosniff, Range. The CI table now links the run rather than only
  its log

Measured through the binary on a real instance: a two-file dist/* declaration ran
on a real runner, both files uploaded over SSH, the run page served them with a
matching sha256, and vole gc --dry-run reported 3 objects, 3 reachable roots (the
log and both artifacts), removing nothing.

internal/gc/gc.go +8−5

@@ -10,8 +10,9 @@
10 10 //
11 11 // - Pointer files in any tree reachable from any ref, in any repository. A
12 12 // pointer names one object by sha256; that is the whole reference.
13 // - Archived CI logs recorded in the run queue. No git ref reaches them, so a
14 // sweep that only marked git would delete every finished log.
13 // - Objects named by the run queue: archived CI logs and published run
14 // artifacts. No git ref reaches them, so a sweep that only marked git would
15 // delete every finished log and every build output a run published.
15 16 //
16 17 // Anything unmarked and younger than the grace period is kept: an upload that
17 18 // lands while the sweep is marking is not reachable yet, but is not garbage
@@ -30,8 +31,10 @@ type Options struct {
30 31 // Repos are bare repository directories.
31 32 Repos []string
32 33 Store *lfs.Store
33 // LogOIDs are objects referenced by run records.
34 LogOIDs []string
34 // RunOIDs are objects named by the run queue: archived logs and published run
35 // artifacts. Neither is reachable from a git ref, so a sweep that only marked
36 // git would delete every finished log and every build output a run published.
37 RunOIDs []string
35 38 // Grace protects recent objects that a concurrent upload has not made
36 39 // reachable yet.
37 40 Grace time.Duration
@@ -61,7 +64,7 @@ func Run(opts Options) (Result, error) {
61 64 opts.Now = time.Now()
62 65 }
63 66 marked := map[string]bool{}
64 for _, oid := range opts.LogOIDs {
67 for _, oid := range opts.RunOIDs {
65 68 marked[oid] = true
66 69 }
67 70