Sign in

Search issue titles and bodies, change requests and comments. Private repositories you cannot read are not included.

vole
Clone

Read only, no credentials:

https://rodent.software/vole.git

Write over SSH with a registered key:

git clone ssh://git@rodent.software:2222/vole.git

A free and open-source forge that lives in your Git repo.

#217

Hold a job's cache namespace while the job runs

merged · by keenan 3d ago · merged into main at 91b47a22 by keenan 3d ago

keenan opened this 3d ago

What

The first slice of #216. A cache namespace is now held for as long as a job may be inside it, and eviction leaves a held namespace for a later pass.

  • ci/cache gains Lock (shared), LockExclusive and TryLockExclusive. The implementation is flock on unix and a process-local RWMutex table elsewhere, following core/git/lock_unix.go / lock_other.go.
  • Trim and Clear attempt the exclusive lock without waiting and skip what they cannot have. Skipped bytes still count against the cap, so the next pass finds them.
  • A runner job takes the lock before it creates the namespace and releases it after the container is gone. --no-cache takes it exclusively and empties the namespace while holding it.
  • The runner no longer trims after every job: the cap is enforced at startup and then on a five-minute timer, so eviction is no longer tied to one job’s lifetime.
  • vole workflow run gets the same treatment, so two local runs in one checkout do not delete each other’s cache.
  • Lock files live under <root>/.locks, mirroring the namespace, and are never deleted — the same rule and reason as gitcache’s. dirs() now skips hidden directories so bookkeeping is never measured or evicted as a cache.

Why this lands on its own

Two runners sharing one --dir already had this bug, and docs/runner.md documents that deployment. The trimmer deletes whole namespaces and --no-cache empties one, either of which could land on a tree another process had mounted. The whole-namespace eviction rule exists precisely because a Go module cache missing one file is a checksum error deep in a build; deleting a live one produces the same class of failure deliberately.

It also corrects an assumption in #216: --no-cache was already scoped to the job’s own namespace, not the whole root. What was missing was the lock, and its Clear at startup was the only whole-root operation.

Decision

Per your comment on #216, /cache stays the runner’s cache: concurrent builds of the same repository share a namespace, and the tool caches’ own locking is what makes that safe. docs/runner.md now says eviction leaves a held namespace for a later pass and that --no-cache empties under that hold.

Verification

  • gofmt -l ., go vet ./..., go build ./... clean.
  • vole workflow check, then vole workflow run verify: ok (format, vet, build, licenses, govulncheck, packages, deployment files, links, and go test -race -count=1 ./...).
  • New tests: trim falls through from a held namespace to the next candidate and takes the held one once released; Clear skips what a job holds; a trimmer looping against two held namespaces removes neither; the lock file outlives its namespace and stays usable; two jobs may share a namespace; --no-cache holds it exclusively; the existing “Cache can be turned off” and local-run cache tests still pass unchanged.

Known and deliberate

  • A lock file remains after its namespace is evicted. Removing it races with the next holder opening it, which is why gitcache keeps its own.
  • On a platform without flock, locks are process-local; the packages are built for the platforms that have it.
  • Namespace contention between different repositories is still the per-repo flock in prepareWorktree; unchanged here.

Refs #216.

Discussion (0)

No discussion yet.