Sign in

Search issue titles and bodies, change requests and comments. Private repositories you cannot read are not included.

vole
Clone

Read only, no credentials:

https://rodent.software/git/vole.git

Write over SSH with a registered key:

git clone ssh://git@0.0.0.0:2222/vole.git

A free and open-source forge that lives in your Git repo.

Hold a job's cache namespace while the job runs

a29ce1f76aaf7ebc4b24afad36c56a0f7f01254e · Keenan Nemetz · 2d ago · parent b492b557

Hold a job's cache namespace while the job runs

A namespace was only safe while one job existed per runner. Two runners
sharing a --dir already had the bug: the trimmer deletes whole namespaces
and `--no-cache` empties one, either of which can land on a tree another
process has mounted. A Go module cache that loses a file mid-build is a
checksum error deep in a build, which is the case the whole-namespace
eviction rule exists to avoid.

A job now holds its namespace for as long as the container may be inside
it — shared, so two builds of the same repository still run together —
and eviction takes the same lock with LOCK_NB and leaves what is in use
for a later pass. `--no-cache` takes the namespace exclusively, because
emptying it afterwards must not land on somebody else's build.

The runner also stops trimming at the end of each job: the cap is
enforced at startup and on a timer, so eviction no longer belongs to one
job's lifetime. Lock files live under <root>/.locks, mirroring the
namespace, so a cache tree holds namespaces and nothing else; they are
never deleted, for the reason gitcache's are not.

`vole workflow run` gets the same lock, so two local runs in one checkout
cannot delete each other's cache either.

Refs: 01M6SCWCSX39QXS38QAHQSMVGX (#216)

docs/runner.md +11−5

@@ -132,7 +132,8 @@ instance or kept on the runner, so each of those channels has a bound:
132 132 most 256 files totalling 4 GiB. The expanded set is checked before anything is
133 133 uploaded; a pattern that matches a directory tree is the usual way to hit this.
134 134 - **Cache.** `--cache-size` (5 GiB by default) caps the cache root, enforced by
135 evicting whole namespaces. Each repository gets its own namespace, and each
135 evicting whole namespaces, and a namespace a job has mounted is left for a later
136 pass. Each repository gets its own namespace, and each
136 137 contributor account gets its own inside a repository, so one contributor's
137 138 branch cannot seed the cache another contributor's branch builds from. A
138 139 protected branch never shares a cache with a contributor branch. The saved
@@ -344,10 +345,15 @@ cache built with one toolchain is not consumed by another.
344 345
345 346 Namespaces are evicted whole and least-recently-written first, so a cache is never
346 347 left half-deleted — a Go module cache missing one file is a checksum error deep in
347 a build rather than a download. `--cache-size` caps the total (5G by default) and
348 the runner trims at startup and after each job. `--no-cache` keeps nothing
349 between jobs: the directory is still mounted — an image that points at `/cache`
350 has to work either way — but it is emptied after every job.
348 a build rather than a download. A job holds its namespace for as long as it runs,
349 and eviction leaves a held namespace for a later pass, so two runners sharing one
350 `--dir` never delete each other's tree in the middle of a build. `--cache-size` caps
351 the total (5G by default), and the runner enforces it at startup and then on its own
352 timer: a cache that is in use when the cap is reached comes down when the job
353 finishes, not under it. `--no-cache` keeps nothing between jobs: the directory is
354 still mounted — an image that points at `/cache` has to work either way — but it is
355 emptied after every job, while that job still holds it, so a shared `--dir` is not
356 emptied under somebody else's build.
351 357
352 358 ## Saved checkouts
353 359