Why
main’s verify run 01M7189M5A5P0PJ170QA8F3CK2 (3229011, “update readme”) failed
its test step. That commit changes only README.md; the failure is in the test
harness, not the code under test.
Two runner defaults interact:
--container-tmpfsdefaults to1g(ci/container/container.go), so the job’s/tmpis a 1 GiB tmpfs.--disk-reservedefaults to2G(cmd/vole/runner.go), checked once a second (ci/agent/agent.go) against the filesystem that holds--dir.
The server integration tests spawn nested runners with --dir t.TempDir()
(server/integration_test.go). t.TempDir() is under /tmp, so those runners
measure a 1 GiB tmpfs against a 2 GiB reserve, stop the job at the first
one-second check, and the test fails:
--- FAIL: TestKilledRunnerIsRequeued (6.55s)
integration_test.go:3437: the retried run did not succeed: failed (exit 1)
the runner has 945.8 MiB free, below its 2.0 GiB reserve;
stopping the job so the machine does not fill (--disk-reserve)
--- FAIL: TestRunLogStreamsWhileTheJobRuns (2.64s)
Only tests whose steps run longer than a second are hit. TestKilledRunnerIsRequeued
holds the lease with sleep 3, and TestRunLogStreamsWhileTheJobRuns streams output
slowly on purpose; the rest of the package finishes each step before the first
check, which is why they pass.
Reproduced locally in the same file:verify image with --tmpfs /tmp:size=1g:
the runner has 769.8 MiB free, below its 2.0 GiB reserve; stopping the job
--- FAIL: TestKilledRunnerIsRequeued (6.55s)
--- FAIL: TestRunLogStreamsWhileTheJobRuns (2.43s)
The reserve protects a host filesystem. A test’s t.TempDir() is a throwaway
directory on a tmpfs the runtime already bounds, so enforcing the host reserve
there proves nothing and makes the harness depend on the runner’s flag defaults.
Scope
- Turn the disk reserve off for the runners the integration tests spawn, at the one place their command line is built.
- Say why in a comment, so a new test does not reintroduce it.
- Confirm the two tests pass with the suite running in a 1 GiB
/tmp.
Acceptance criteria
TestKilledRunnerIsRequeuedandTestRunLogStreamsWhileTheJobRunspass whengo test ./serverruns in a job container whose/tmpis a 1 GiB tmpfs.go test -race -count=1 ./serverpasses locally andvole workflow run verifypasses in the verify image.- No production behavior changes: the runner keeps the 2 GiB
--disk-reservedefault, andci/agent/disk_test.gostill covers the check itself.