Sign in

Search issue titles and bodies, change requests and comments. Private repositories you cannot read are not included.

vole
Clone

Read only, no credentials:

https://rodent.software/git/vole.git

Write over SSH with a registered key:

git clone ssh://git@0.0.0.0:2222/vole.git

A free and open-source forge that lives in your Git repo.

Phase 4: a job can name the machine it needs

6c03e461ee409f4bd053ee552edd87b6188a5eb2 · Keenan Nemetz · 16d ago · parent b238ac24

Phase 4: a job can name the machine it needs

`runs-on: [linux, arm64]` on a job, `--label linux` on a runner, and the
lease hands a job only to a runner carrying every label it asked for. Extra
labels on the runner are not a mismatch — the rule is set inclusion, so a
machine more specific than the job needs still qualifies — and a job with no
runs-on matches every runner, which is what every job did before this
existed.

Both spellings are accepted (`runs-on: linux` and a list), because both are
the obvious way to write it and a dialect already separate from GitHub's
should not also fail on the spelling a person guessed. Labels are validated
in pre-receive by the same rule the runner applies to --label at startup: a
label no runner can carry leaves a job queued forever and looks exactly like
a busy pool, which is the failure nobody debugs because there is no error.

Labels are pinned at enqueue next to the steps and the artifact patterns. A
workflow edited after a push must not change where an already-queued job is
allowed to run. That also settles the derived-store question: run_labels is
queue state, not history, because a finished run's requirement is recoverable
from the pinned workflow blob, so `vole index rebuild` does not restore
labels and nothing needs it to.

A label is a claim by the runner about itself, unlike --repos, which the
server enforces. A runner that lies about its architecture only takes work it
will do badly; a runner that could name repositories it was not granted would
be an access-control hole. /runners now shows what each connected runner
takes, and says "any job" for an unlabelled one — the other half of "why has
my build not started", since queue depth alone cannot say that no runner
carrying the needed label is connected.

Enqueue takes labels as a second argument rather than a field on Run: they
are lease-time policy, not part of what a run is, and every reader of a Run
(a gate, a page, a record rebuild) would be looking at a field only a lease
ever consulted.

Measured: TestRunsOnWaitsForARunnerThatCarriesTheLabel — a runner carrying
only `linux` leaves a `[linux, fast]` job queued and the runner carrying both
builds it. TestLabelsDecideWhichRunnerMayTakeAJob covers no requirement,
exact match, one label missing, wrong labels and extra labels;
TestLabelsArePinnedAtEnqueue, TestForgettingARepositoryDropsItsLabels,
TestABadRunnerLabelIsRefusedBeforeConnecting,
TestRunsOnLabelsAreParsedAndValidated and
TestRunnersPageShowsWhatEachRunnerTakes cover the edges.

cmd/vole/runner.go +43−12

@@ -21,6 +21,7 @@ import (
21 21 "github.com/rodent-software/vole/internal/lfs"
22 22 "github.com/rodent-software/vole/internal/releases"
23 23 "github.com/rodent-software/vole/internal/runs"
24 "github.com/rodent-software/vole/internal/workflow"
24 25 "golang.org/x/crypto/ssh"
25 26 "sync"
26 27 "syscall"
@@ -34,7 +35,7 @@ const leaseWait = 25 * time.Second
34 35
35 36 func runnerUsage() string {
36 37 return `usage: vole runner --remote ssh://git@host[:port] [--once] [--dir path] [--key path]
37 [--repos name]...
38 [--repos name]... [--label name]...
38 39
39 40 Leases CI jobs from a Vole instance and runs them. Authentication is the SSH key
40 41 you would push with, so there is no token to configure. A runner may only run jobs
@@ -55,7 +56,11 @@ repository's runner allowlist (vole acl runners) cannot report at all.
55 56 $GIT_SSH_COMMAND, else ~/.ssh/id_ed25519)
56 57 --repos only take jobs for these repositories (repeatable). It narrows what the
57 58 account's runner allowlist grants; it can never add to it. Without it a
58 runner takes any repository it is allowed to run.`
59 runner takes any repository it is allowed to run.
60 --label what this machine is, e.g. linux or arm64 (repeatable). A workflow asks
61 for a machine with runs-on, and a job is only given to a runner carrying
62 every label it asks for. Without labels a runner only takes jobs that ask
63 for nothing, which is what every job did before runs-on existed.`
59 64 }
60 65
61 66 type runnerStep struct {
@@ -113,6 +118,9 @@ type runnerRequest struct {
113 118 // Repos is the --repos filter, sent with every lease. The server intersects it
114 119 // with what the account's allowlists grant, so it can only narrow.
115 120 Repos []string `json:"repos,omitempty"`
121 // Labels are what this machine is, sent with every lease so the server can match
122 // them against the runs-on a job asked for.
123 Labels []string `json:"labels,omitempty"`
116 124 // Wait is how many seconds an idle lease may park before answering "nothing to
117 125 // do"; the server caps it. It is what makes an idle runner cost a parked channel
118 126 // rather than a query every couple of seconds.
@@ -142,7 +150,8 @@ type runnerReply struct {
142 150 // is: no separate credential, nothing stored.
143 151 func cmdRunner(args []string) error {
144 152 f, err := parseFlags(args, flagSpec{long: "remote", short: "r"}, flagSpec{long: "dir"},
145 flagSpec{long: "key"}, flagSpec{long: "repos"}, flagSpec{long: "once", boolean: true})
153 flagSpec{long: "key"}, flagSpec{long: "repos"}, flagSpec{long: "label"},
154 flagSpec{long: "once", boolean: true})
146 155 if err != nil {
147 156 return err
148 157 }
@@ -166,6 +175,15 @@ func cmdRunner(args []string) error {
166 175 return err
167 176 }
168 177
178 // A label has to be one the workflow validator would accept, or the machine would
179 // advertise something no job can ask for — a typo with no symptom at all.
180 labels := f.all("label")
181 for _, label := range labels {
182 if err := workflow.ValidLabel(label); err != nil {
183 return fmt.Errorf("--label %w", err)
184 }
185 }
186
169 187 sess, err := dialRunner(remote)
170 188 if err != nil {
171 189 return err
@@ -199,20 +217,14 @@ func cmdRunner(args []string) error {
199 217 return err
200 218 }
201 219 sess.account, sess.name, sess.fingerprint = who.Account, who.Name, fingerprint
202 sess.repos = f.all("repos")
203 if len(sess.repos) > 0 {
204 fmt.Fprintf(os.Stderr, "vole runner: connected as %s, signing with %s, only taking %s; waiting for work\n",
205 who.Name, fingerprint, strings.Join(sess.repos, ", "))
206 } else {
207 fmt.Fprintf(os.Stderr, "vole runner: connected as %s, signing with %s; waiting for work\n",
208 who.Name, fingerprint)
209 }
220 sess.repos, sess.labels = f.all("repos"), labels
221 fmt.Fprintf(os.Stderr, "vole runner: %s\n", sess.describe(who.Name, fingerprint))
210 222 for {
211 223 // A resident runner parks on the server's lease: the reply arrives when work is
212 224 // queued, not on a fixed beat, so an idle pool costs one open channel per
213 225 // runner instead of a query every couple of seconds. A one-shot runner asks
214 226 // once and leaves.
215 req := runnerRequest{Op: "lease", Repos: sess.repos}
227 req := runnerRequest{Op: "lease", Repos: sess.repos, Labels: sess.labels}
216 228 if !f.has("once") {
217 229 req.Wait = int(leaseWait.Seconds())
218 230 }
@@ -269,6 +281,25 @@ type runnerSession struct {
269 281 // asked for, not the effective list: the server intersects it with the grant on
270 282 // each request, so an allowlist edit takes effect without a restart.
271 283 repos []string
284 // labels is what this machine is (--label). A job is only given to a runner that
285 // carries every label the job asked for, so an unlabelled runner only sees jobs
286 // that ask for nothing.
287 labels []string
288 }
289
290 // describe is the one line a runner prints when it starts work, because "why is
291 // nothing running" is answered by reading what it says it will take. Both filters are
292 // named when they are set and omitted when they are not, so the common case stays one
293 // short line.
294 func (s *runnerSession) describe(name, fingerprint string) string {
295 out := "connected as " + name + ", signing with " + fingerprint
296 if len(s.labels) > 0 {
297 out += ", taking jobs that ask for " + strings.Join(s.labels, ", ")
298 }
299 if len(s.repos) > 0 {
300 out += ", only in " + strings.Join(s.repos, ", ")
301 }
302 return out + "; waiting for work"
272 303 }
273 304
274 305 // checkRepoFilter refuses a --repos filter that can never match. A name the server