Sign in

Search issue titles and bodies, change requests and comments. Private repositories you cannot read are not included.

vole
Clone

Read only, no credentials:

https://rodent.software/git/vole.git

Write over SSH with a registered key:

git clone ssh://git@0.0.0.0:2222/vole.git

A free and open-source forge that lives in your Git repo.

Phase 4: a job can name the machine it needs

6c03e461ee409f4bd053ee552edd87b6188a5eb2 · Keenan Nemetz · 16d ago · parent b238ac24

Phase 4: a job can name the machine it needs

`runs-on: [linux, arm64]` on a job, `--label linux` on a runner, and the
lease hands a job only to a runner carrying every label it asked for. Extra
labels on the runner are not a mismatch — the rule is set inclusion, so a
machine more specific than the job needs still qualifies — and a job with no
runs-on matches every runner, which is what every job did before this
existed.

Both spellings are accepted (`runs-on: linux` and a list), because both are
the obvious way to write it and a dialect already separate from GitHub's
should not also fail on the spelling a person guessed. Labels are validated
in pre-receive by the same rule the runner applies to --label at startup: a
label no runner can carry leaves a job queued forever and looks exactly like
a busy pool, which is the failure nobody debugs because there is no error.

Labels are pinned at enqueue next to the steps and the artifact patterns. A
workflow edited after a push must not change where an already-queued job is
allowed to run. That also settles the derived-store question: run_labels is
queue state, not history, because a finished run's requirement is recoverable
from the pinned workflow blob, so `vole index rebuild` does not restore
labels and nothing needs it to.

A label is a claim by the runner about itself, unlike --repos, which the
server enforces. A runner that lies about its architecture only takes work it
will do badly; a runner that could name repositories it was not granted would
be an access-control hole. /runners now shows what each connected runner
takes, and says "any job" for an unlabelled one — the other half of "why has
my build not started", since queue depth alone cannot say that no runner
carrying the needed label is connected.

Enqueue takes labels as a second argument rather than a field on Run: they
are lease-time policy, not part of what a run is, and every reader of a Run
(a gate, a page, a record rebuild) would be looking at a field only a lease
ever consulted.

Measured: TestRunsOnWaitsForARunnerThatCarriesTheLabel — a runner carrying
only `linux` leaves a `[linux, fast]` job queued and the runner carrying both
builds it. TestLabelsDecideWhichRunnerMayTakeAJob covers no requirement,
exact match, one label missing, wrong labels and extra labels;
TestLabelsArePinnedAtEnqueue, TestForgettingARepositoryDropsItsLabels,
TestABadRunnerLabelIsRefusedBeforeConnecting,
TestRunsOnLabelsAreParsedAndValidated and
TestRunnersPageShowsWhatEachRunnerTakes cover the edges.

internal/workflow/workflow_test.go +73−1

@@ -1,6 +1,7 @@
1 1 package workflow
2 2
3 3 import (
4 "strconv"
4 5 "strings"
5 6 "testing"
6 7 "time"
@@ -72,7 +73,7 @@ func TestInvalidWorkflowsRejected(t *testing.T) {
72 73 {"bad job id", "version: 1\njobs:\n Bad Job:\n steps:\n - run: echo\n", "job names are"},
73 74 {"bad timeout", "version: 1\njobs:\n j:\n steps:\n - run: echo\n timeout: soon\n", "not a duration"},
74 75 {"huge timeout", "version: 1\njobs:\n j:\n steps:\n - run: echo\n timeout: 5h\n", "exceeds"},
75 {"typo key", "version: 1\njobs:\n j:\n runs-on: x\n steps:\n - run: echo\n", "is not a known key"},
76 {"typo key", "version: 1\njobs:\n j:\n run-on: x\n steps:\n - run: echo\n", "is not a known key"},
76 77 {"empty", "", "empty"},
77 78 {"bad glob", "version: 1\non:\n push:\n branches: [\"a b\"]\njobs:\n j:\n steps:\n - run: echo\n", "not a branch pattern"},
78 79 }
@@ -296,3 +297,74 @@ jobs:
296 297 t.Fatal("a tag pattern with .. was accepted")
297 298 }
298 299 }
300
301 // runs-on names the machine a job needs. Both spellings are accepted, the labels are
302 // normalised, and a label no runner could ever carry fails the push rather than
303 // leaving the job queued forever behind an error nobody sees.
304 func TestRunsOnLabelsAreParsedAndValidated(t *testing.T) {
305 // A job that says nothing keeps the pre-targeting behaviour: any runner.
306 wf, problems := Parse("ci.yaml", `version: 1
307 jobs:
308 build:
309 steps:
310 - run: true
311 `)
312 if len(problems) > 0 {
313 t.Fatalf("problems: %v", problems)
314 }
315 if len(wf.Jobs[0].Labels) != 0 {
316 t.Fatalf("labels = %v, want none", wf.Jobs[0].Labels)
317 }
318
319 // One label as a scalar, several as a list.
320 for _, src := range []string{
321 "version: 1\njobs:\n build:\n runs-on: linux\n steps:\n - run: true\n",
322 "version: 1\njobs:\n build:\n runs-on: [linux]\n steps:\n - run: true\n",
323 } {
324 wf, problems := Parse("ci.yaml", src)
325 if len(problems) > 0 {
326 t.Fatalf("problems for %q: %v", src, problems)
327 }
328 if len(wf.Jobs[0].Labels) != 1 || wf.Jobs[0].Labels[0] != "linux" {
329 t.Fatalf("labels = %v, want [linux]", wf.Jobs[0].Labels)
330 }
331 }
332
333 wf, problems = Parse("ci.yaml", `version: 1
334 jobs:
335 build:
336 runs-on: [linux, arm64, linux]
337 steps:
338 - run: true
339 `)
340 if len(problems) > 0 {
341 t.Fatalf("problems: %v", problems)
342 }
343 // A repeated label is asked for once.
344 if got := strings.Join(wf.Jobs[0].Labels, ","); got != "linux,arm64" {
345 t.Fatalf("labels = %q, want linux,arm64", got)
346 }
347
348 // A label that can never be a runner's label is refused, and the message names the
349 // rule rather than only the value.
350 for _, bad := range []string{"Linux", "-x", "a b", ""} {
351 src := "version: 1\njobs:\n build:\n runs-on: [" + strconv.Quote(bad) + "]\n" +
352 " steps:\n - run: true\n"
353 if _, problems := Parse("ci.yaml", src); len(problems) == 0 {
354 t.Fatalf("runs-on label %q was accepted", bad)
355 } else if !strings.Contains(strings.Join(problems, " "), "runs-on") {
356 t.Fatalf("the problem does not name runs-on: %v", problems)
357 }
358 }
359
360 // Too many labels is describing a machine, which is what --repos is for.
361 many := make([]string, MaxLabels+1)
362 for i := range many {
363 many[i] = "l" + strconv.Itoa(i)
364 }
365 src := "version: 1\njobs:\n build:\n runs-on: [" + strings.Join(many, ", ") + "]\n" +
366 " steps:\n - run: true\n"
367 if _, problems := Parse("ci.yaml", src); len(problems) == 0 {
368 t.Fatal("a job with too many labels was accepted")
369 }
370 }