Part 2 of #11, on top of the merged #56. Closes #11 with it.
What was wrong
A job runs untrusted code, and nothing bounded what it could make the runner or
the instance keep: the log was a strings.Builder for the whole attempt (then a
temp file, then read twice by the transfer), an artifact pattern could expand to
a whole tree, all contributor branches shared one cache namespace, and a host
path went into a --volume field unchecked.
What this does
Bounded log, uploaded once. ci/agent/spool.go writes the attempt’s log to a
capped file from the first byte (32 MiB) and the transfer uploads that file. No
whole-attempt buffer, no second copy, no second read. A capped log still reports
the whole write so the producing process is not killed, and gets a line saying it
was truncated. Step offsets come from the spool’s length, which never exceeds the
cap, so a truncated log’s ranges stay valid against the uploaded object.
Nothing is streamed to the server. A log’s content-addressed id cannot be known before its last byte, so streaming would need a second protocol; the only shape consistent with “LFS only” is chunk objects uploaded as ordinary content-addressed objects during the run, which is a record and protocol change with its own decisions (live logs, partial logs from dead runners) and belongs in a separate issue.
Artifact caps. 256 files and 4 GiB per run, checked after the patterns are
expanded and before anything is uploaded. ci.MaxArtifacts bounds declarations; a
declaration is a pattern that can match a tree, so the expanded set needed its own
bound.
Contributor caches by account. The contrib namespace becomes
contrib-<account>. One contributor’s untrusted branch can no longer seed
another’s cache, and a contributor keeps their cache across their own branches.
Volume path validation. container.ValidatePaths refuses a work or mount path
containing :, NUL, CR or LF before the argument list is built, since : is the
field separator in --volume. Start calls it; RunArgs is unchanged. The
closed-set test now also pins --cap-add, --device, --pid/--ipc host and the
Podman socket.
Image builds documented. Built by the configured runtime, so rootless Podman
builds rootless; the definition is already pinned by digest (ImageSHA), and the
docs now explain that the build uses the runtime’s network and recommend pinning
the base image in from: by digest for reproducibility.
Acceptance
- A malicious payload (
repo=../../…,pages.path=../../…, bad SHA) is refused by part 1’svalidateJob; symlinks are already skipped as artifacts, and a climbing or absolute artifact pattern is refused by the shared validator. - Memory and disk are bounded: log 32 MiB, artifacts 256 files / 4 GiB, cache 5 GiB by default, container limits from the runtime flags.
Verification
gofmt,go vet ./...,go test -race ./...(CI packages), full./vole workflow checkand./vole workflow run verifyclean — including the end-to-end runner tests, which exercise the new log path through a real upload.
Tests
TestSpoolKeepsWhatFits,TestSpoolTruncatesAndSaysSo(cap, note,Lenstays within the cap),TestSpoolCloseRemovesTheFile.TestCheckArtifactLimits(count, and bytes via a sparse 4 GiB file).TestContributorCachesAreSeparatedByAccount(two contributors differ, one contributor’s branches share).TestInvalidVolumePathsAreRefusedand the extendedTestRunArgsAreClosed.
Docs: docs/runner.md (“What a job may consume”, image-build network and
rootless notes), docs/reference/workflow-reference.md (expanded bounds),
docs/reference/image-reference.md (from: digest pinning).