vole
Clone
Read only, no credentials:
https://rodent.software/git/vole.git
Write over SSH with a registered key:
git clone ssh://git@0.0.0.0:2222/vole.git
A free and open-source forge that lives in your Git repo.
Phase 4: an idle runner parks on the queue instead of polling it
34afc5691bf2d6c808f905935cc95ffba34f09a7 · Keenan Nemetz · 16d ago · parent a945ae55
Phase 4: an idle runner parks on the queue instead of polling it Two costs made a runner pool more expensive than it needed to be. Every lease request re-derived the account's eligible repositories by opening every repository and reading its rules — O(repos) per idle runner, every two seconds, to be told there was nothing to do. And the "long poll" was the client's own timer: the queue answered at once and the runner slept. The lease now takes a `wait`: the session holds the request on a wake-up channel that a committed enqueue closes (and RequeueExpired closes when it puts a job back), wakes and asks once more, because the wake-up may belong to a job another runner took first. The channel is taken *before* the lease attempt, so a run queued between the two cannot be missed. It is in-process on purpose: one `vole serve` owns this database, and the wait is a courtesy, never a durability mechanism — a lost wake-up costs one poll interval and nothing else. The eligibility list is now reused for the roster's five-second window instead of being re-derived per request, so an allowlist change still takes effect without a reconnect but a quiet pool makes no queries at all. `--once` sends no wait: a one-shot runner is here for the job it was started for. Measured: TestALeaseWaitsForWorkInsteadOfPolling parks a lease for 30s and gets the job inside 5s of it being queued; TestALeaseWithoutAWaitAnswersAtOnce returns from an empty queue immediately; TestQueuingWorkWakesTheParked and TestADuplicateEnqueueDoesNotWakeTheParked pin that a queued run wakes waiters and a duplicate does not. Also corrected: `vole runner --help` claimed jobs come from repositories the account administers, which stopped being true when `acl runners` became the allowlist.
cmd/vole/runner.go +25−2
| @@ -26,12 +26,19 @@ import ( | |||
| 26 | 26 | "time" | |
| 27 | 27 | ) | |
| 28 | 28 | ||
| 29 | // leaseWait is how long an idle runner parks on the lease before asking again. It is | ||
| 30 | // below the 60-second cap the server applies and long enough that a quiet pool is | ||
| 31 | // invisible in the queue's load. | ||
| 32 | const leaseWait = 25 * time.Second | ||
| 33 | |||
| 29 | 34 | func runnerUsage() string { | |
| 30 | 35 | return `usage: vole runner --remote ssh://git@host[:port] [--once] [--dir path] [--key path] | |
| 31 | 36 | ||
| 32 | 37 | Leases CI jobs from a Vole instance and runs them. Authentication is the SSH key | |
| 33 | 38 | you would push with, so there is no token to configure. A runner may only run jobs | |
| 34 | for repositories where its account has admin. | ||
| 39 | for repositories whose runner allowlist its account is on (admins are implicit): | ||
| 40 | running a job executes that repository's code, which is a bigger power than | ||
| 41 | writing to it. | ||
| 35 | 42 | ||
| 36 | 43 | Every finished attempt is signed with --key and committed to the repository's | |
| 37 | 44 | metadata branch, together with the artifacts the build published, in one push: the | |
| @@ -98,6 +105,10 @@ type runnerArtifact struct { | |||
| 98 | 105 | type runnerRequest struct { | |
| 99 | 106 | Op string `json:"op"` | |
| 100 | 107 | Run string `json:"run"` | |
| 108 | // Wait is how many seconds an idle lease may park before answering "nothing to | ||
| 109 | // do"; the server caps it. It is what makes an idle runner cost a parked channel | ||
| 110 | // rather than a query every couple of seconds. | ||
| 111 | Wait int `json:"wait,omitempty"` | ||
| 101 | 112 | } | |
| 102 | 113 | ||
| 103 | 114 | type runnerReply struct { | |
| @@ -172,7 +183,15 @@ func cmdRunner(args []string) error { | |||
| 172 | 183 | fmt.Fprintf(os.Stderr, "vole runner: connected as %s, signing with %s; waiting for work\n", | |
| 173 | 184 | who.Name, fingerprint) | |
| 174 | 185 | for { | |
| 175 | rep, err := sess.call(runnerRequest{Op: "lease"}) | ||
| 186 | // A resident runner parks on the server's lease: the reply arrives when work is | ||
| 187 | // queued, not on a fixed beat, so an idle pool costs one open channel per | ||
| 188 | // runner instead of a query every couple of seconds. A one-shot runner asks | ||
| 189 | // once and leaves. | ||
| 190 | req := runnerRequest{Op: "lease"} | ||
| 191 | if !f.has("once") { | ||
| 192 | req.Wait = int(leaseWait.Seconds()) | ||
| 193 | } | ||
| 194 | rep, err := sess.call(req) | ||
| 176 | 195 | if err != nil { | |
| 177 | 196 | return err | |
| 178 | 197 | } | |
| @@ -183,6 +202,10 @@ func cmdRunner(args []string) error { | |||
| 183 | 202 | if f.has("once") { | |
| 184 | 203 | return nil | |
| 185 | 204 | } | |
| 205 | // Either the wait expired with nothing to do, or the server answered at | ||
| 206 | // once because it is older than this client and ignores `wait`. The sleep | ||
| 207 | // keeps the second case at a polite two requests per second instead of a | ||
| 208 | // spin. | ||
| 186 | 209 | time.Sleep(2 * time.Second) | |
| 187 | 210 | continue | |
| 188 | 211 | } | |