vole

Clone

Read only, no credentials:

https://rodent.software/vole.git

Write over SSH with a registered key:

git clone ssh://git@rodent.software:2222/vole.git

A free and open-source forge that lives in your Git repo.

Hold a job's cache namespace while the job runs

a29ce1f76aaf7ebc4b24afad36c56a0f7f01254e · Keenan Nemetz · 3d ago · parent b492b557

Hold a job's cache namespace while the job runs

A namespace was only safe while one job existed per runner. Two runners
sharing a --dir already had the bug: the trimmer deletes whole namespaces
and `--no-cache` empties one, either of which can land on a tree another
process has mounted. A Go module cache that loses a file mid-build is a
checksum error deep in a build, which is the case the whole-namespace
eviction rule exists to avoid.

A job now holds its namespace for as long as the container may be inside
it — shared, so two builds of the same repository still run together —
and eviction takes the same lock with LOCK_NB and leaves what is in use
for a later pass. `--no-cache` takes the namespace exclusively, because
emptying it afterwards must not land on somebody else's build.

The runner also stops trimming at the end of each job: the cap is
enforced at startup and on a timer, so eviction no longer belongs to one
job's lifetime. Lock files live under <root>/.locks, mirroring the
namespace, so a cache tree holds namespaces and nothing else; they are
never deleted, for the reason gitcache's are not.

`vole workflow run` gets the same lock, so two local runs in one checkout
cannot delete each other's cache either.

Refs: 01M6SCWCSX39QXS38QAHQSMVGX (#216)

ci/cache/lock_other.go +46

@@ -0,0 +1,46 @@
	1	//go:build !unix
	2
	3	package cache
	4
	5	import (
	6	"os"
	7	"sync"
	8	)
	9
	10	// flock is not part of the portable API, so a platform without it serialises
	11	// holders inside one process only. Vole ships runner packages for the platforms
	12	// where the kernel lock exists; this keeps the rest building, and says what it
	13	// does: a job and the trimmer in one process are excluded from each other, and a
	14	// second process on the same --dir is not.
	15	var (
	16	processLockMu sync.Mutex
	17	processLocks = map[string]*sync.RWMutex{}
	18	)
	19
	20	func lockFile(f *os.File, exclusive, wait bool) (func(), bool, error) {
	21	processLockMu.Lock()
	22	mu := processLocks[f.Name()]
	23	if mu == nil {
	24	mu = &sync.RWMutex{}
	25	processLocks[f.Name()] = mu
	26	}
	27	processLockMu.Unlock()
	28	switch {
	29	case exclusive && !wait:
	30	if !mu.TryLock() {
	31	return nil, false, nil
	32	}
	33	return mu.Unlock, true, nil
	34	case exclusive:
	35	mu.Lock()
	36	return mu.Unlock, true, nil
	37	case !wait:
	38	if !mu.TryRLock() {
	39	return nil, false, nil
	40	}
	41	return mu.RUnlock, true, nil
	42	default:
	43	mu.RLock()
	44	return mu.RUnlock, true, nil
	45	}
	46	}