Sign in

Search issue titles and bodies, change requests and comments. Private repositories you cannot read are not included.

vole
Clone

Read only, no credentials:

https://rodent.software/git/vole.git

Write over SSH with a registered key:

git clone ssh://git@0.0.0.0:2222/vole.git

A free and open-source forge that lives in your Git repo.

Record a job that never started as could-not-run, not as a failure

70c9ad4eb3f3d0cd39159e647cde77afba75b944 · Keenan Nemetz · 2h ago · parent 9704d735

Record a job that never started as could-not-run, not as a failure

A run that never executed a step was recorded exit=1, steps=nil, and the run page
said the same thing it says when a test fails. With checks required on a protected
branch, a runner that could not build an image red-flagged a commit whose tests
pass, and the only way back was a rerun by hand.

Two facts were collapsed into one: the job ran and something failed, which is the
repository's problem, and the job never ran, which is the machine's. The runner
knows the difference exactly once, and it does not need to guess whose fault it
was: if no step ran, nothing about the commit was tested.

- A record can now say where it stopped: Failure=setup means the attempt ended
  before the job's first step, with the runner's own one-line reason in
  FailureMessage, masked the way the log is. The reason is what the page shows
  instead of making a reader open the log to learn the machine broke.
- A run whose newest attempt has that origin is StatusError, which the pages call
  "could not run". It is not StatusFailed: it is not evidence against the commit.
- The gate reads it as a wait, not a refusal. Landing stays blocked because nothing
  is verified, but the commit is not blamed, and the gap names the reason and the
  rerun command.
- The run list counts and filters it, the run page explains it, and the change list
  says "CI could not run" rather than "CI failed".

A step that exits non-zero is still a failure and still reds the check, because
that one is the code's. `vole runner` still reports a setup failure with exit 1;
the origin is what carries the meaning, so an old runner and an old instance keep
talking.

Not done here, on purpose: a run whose attempts all vanished before reporting (the
lease-loss exhaustion in RequeueExpired) still ends as failed. That run has no
record to say what ran, and a job that repeatedly kills its runner is not
distinguishable from a machine problem from the queue's side; issue #243 is where
that gets a signal. Nor is a setup failure requeued automatically: MaxAttempts is
about flaky tests, not broken runners, and #247 already stops a runner whose
runtime cannot start a container from serving at all. A rerun is a button.

The finish path now takes an attemptOutcome instead of six positional arguments,
which is also what lets a failure after the last step keep the steps that ran
instead of dropping them.

Closes 01M6Z05K5PM58EE3H6VE2P2EYD

docs/runner.md +4

@@ -510,6 +510,10 @@ How the queue behaves:
510 510 `canceled` rather than becoming a failure, and its page says who asked.
511 511 - If a runner disappears, its job is requeued. After five attempts the job fails
512 512 instead of cycling forever.
513 - A job whose attempt stops before its first step — the image build, the container,
514 the network — is recorded as `error` (the page says **could not run**) with the
515 runner's reason, not as a failure of the commit. A required check waits on it
516 rather than going red; rerunning the job clears it.
513 517 - Deleting `data/runs.db` loses CI history and attempts that were in flight. Current
514 518 branch and tag tips are queued again from Git, but only a backup can restore the
515 519 finished runs.