Sign in

Search issue titles and bodies, change requests and comments. Private repositories you cannot read are not included.

vole
Clone

Read only, no credentials:

https://rodent.software/git/vole.git

Write over SSH with a registered key:

git clone ssh://git@0.0.0.0:2222/vole.git

A free and open-source forge that lives in your Git repo.

#219

Build one image-file digest once per runner process

merged · by keenan 2d ago · merged into main at 77056ba8 by keenan 2d ago

keenan opened this 2d ago

What

Slice 3 of #216: one image-file digest is built once per runner process.

Two slots that lease jobs naming the same digest used to build it twice — same bytes, same tag, twice the CPU and twice the disk, and two runtimes racing to tag the result. The first slot builds; the rest wait on the same answer.

  • imageBuilds lives on runnerBox, so it is shared by every slot in the process (that is what --jobs creates) and nothing outside it.
  • A waiter heartbeats on the same beat its own build would have. A build can run for half an hour, and a runner that goes quiet loses the run it is holding; without this, the waiter would be requeued mid-wait and then run a job nobody was waiting for. It stops if the lease is gone (errLeaseLost) or a maintainer cancels the run (errCanceled), both reported the way a build already reports them.
  • A waiter says so at the top of its own log, so a reader knows why the steps have not started.
  • A failed build fails the jobs waiting on it, with the builder’s reason: the same definition would fail the same way, and a second attempt only spends the time again.
  • A finished entry is forgotten, so the next job asks the runtime again — which is also how an image somebody deleted gets rebuilt.

Across processes it can still happen twice. Two runners sharing a --dir will both build the digest. That is waste, not damage: the definition is pinned by digest, so both produce the same image under the same tag; serialising it would need a lock file per digest beside the caches. Noted in the source comment and here rather than built.

Tests

ci/agent/build_test.go, against the existing fake runtime (extracted into fakeImageRuntime, taught to sleep and to fail):

  • TestTwoJobsBuildOneImage: two goroutines, two sessions, one digest, a build slow enough for them to overlap — one build, one tag, and one of them logged that it was waiting.
  • TestAFailedBuildFailsTheJobsWaitingOnIt.
  • TestImageBuildsJoinAndLeave: one builder at a time, a different digest is a different build, a forgotten digest is rebuilt.
  • TestWaitForBuildEnds: the three ways a wait ends — build finished, lease gone, run canceled (and the log names who canceled it).
  • TestResolveImageBuildsAndCachesDefinition: unchanged behaviour at one job.

Verification

gofmt -l ., go vet ./..., go test -tags sqlite_fts5 ./... clean, and vole workflow run verify ok (format, vet, build, licenses, govulncheck, packages, deployment files, links, go test -race -count=1 ./...).

Where #216 stands

Slices 1–3 are in: cache namespace locks and the trim timer (#217), --jobs (#218), and this. The docs acceptance item is covered by #218 (docs/runner.md, docs/cli.md, docs/deploy/runner.md, packaging/runner.env) plus the paragraph this change adds.

What is left on #216 is item 6’s second half: SIGTERM should drain rather than kill in-flight jobs. With --jobs a kill costs N jobs instead of one. That needs signal.NotifyContext, a way to cut a parked lease short, and a second signal that still kills the process — its own change, and the one I would take next.

Also untested end to end: --jobs 4 against a real instance with a container runtime. The session half is covered (TestSeveralSessionsLeaseDifferentJobs), the slot half by unit tests, but nobody has watched four jobs run at once on a machine yet.

Refs #216 (does not close it).

Discussion (0)

No discussion yet.