vole
Clone
Read only, no credentials:
https://rodent.software/vole.git
Write over SSH with a registered key:
git clone ssh://git@rodent.software:2222/vole.git
A free and open-source forge that lives in your Git repo.
Phase 4: the runner signs the result it reports
22528ac3da576387a8311969641a7b5783b04924 · Keenan Nemetz · 17d ago · parent 86fe5bb4
Phase 4: the runner signs the result it reports
An instance-key signature is the server witnessing itself. It cannot be checked
without trusting the box that produced it, and it says nothing about the party that
ran the steps — so a record signed that way would have been paperwork. The runner
already holds a key, the one it connects with, so the runner is what signs.
At finish the runner signs a claim: run, repo, ref, SHA, workflow blob, the digest of
the steps pinned when the job was queued, exit code, the log's {oid, size}, the
artifacts it published, and its own finishing time. The server rebuilds the same claim
from its own row plus what the runner reports, verifies the signature, and stores the
payload and signature verbatim — so a verifier re-checks the bytes that were signed
rather than re-deriving them from a format that might have drifted.
Who the signer is comes from the channel, not the claim: the account is the one the
SSH key resolved to, and the signature must verify under one of that account's keys.
A runner may therefore sign with a different key from the one it connected with — an
agent-held key to connect, a file to sign — but not as somebody else.
Refusals, all before anything is recorded, because half a record is worse than none:
an unsigned finish (an unattributed result is what the queue already had), a signature
by another account's key, and a signature over a different claim. The run page shows
`attested by <name> · key <fingerprint>`, and `vole runner` now pings first and refuses
to start work when the key it will sign with is not one its account holds — discovering
that after a ten-minute build is a wasted build.
`steps_sha` is recorded at enqueue and travels with the lease, so the claim covers the
steps that ran and not merely the workflow file they came from.
What this does not buy: truth. A runner can still lie about an exit code. It buys
accountability — the record is attributable to a party other than the instance.
Whether a required check must be attested by a particular runner (an ACL key defaulting
to the instance's own) is the next policy question and is deliberately not answered
here; any admin of the repository may still supply the signature.
Verified: TestAFinishedRunBelongsToTheRunnerThatSignsIt drives the real protocol over
SSH — the lease carries what the claim names, ping names the account and its keys,
unsigned and wrong-account and wrong-claim finishes are all refused with nothing
recorded, the correct one is accepted, and the stored payload still verifies under the
stored signature; TestClaimRendersOneWay pins the byte form (including artifact
ordering and an empty log) and TestClaimRefusesToForgeItself refuses a field carrying a
newline; TestStepDigestFollowsTheSteps shows the digest covering the steps themselves;
the cmd tests cover key discovery from VOLE_SSH_COMMAND/GIT_SSH_COMMAND, the missing-key
message, a non-key file, and the pre-work refusal; and the end-to-end test that runs the
real binary asserts the claim names the run, its log and its exit, and that the run page
says who attested it. scripts/ci.sh green (gofmt, vet, build, markdown links, go test
-race ./...).
cmd/vole/runner.go +169−16
| @@ -3,6 +3,7 @@ package main | |||
| 3 | 3 | import ( | |
| 4 | 4 | "bufio" | |
| 5 | 5 | "context" | |
| 6 | "crypto/ed25519" | ||
| 6 | 7 | "encoding/json" | |
| 7 | 8 | "errors" | |
| 8 | 9 | "fmt" | |
| @@ -11,24 +12,34 @@ import ( | |||
| 11 | 12 | "os/exec" | |
| 12 | 13 | "path" | |
| 13 | 14 | "path/filepath" | |
| 15 | "slices" | ||
| 14 | 16 | "sort" | |
| 15 | 17 | "strings" | |
| 16 | 18 | ||
| 17 | 19 | "github.com/rodent-software/vole/internal/releases" | |
| 20 | "github.com/rodent-software/vole/internal/runs" | ||
| 21 | "golang.org/x/crypto/ssh" | ||
| 18 | 22 | "sync" | |
| 19 | 23 | "syscall" | |
| 20 | 24 | "time" | |
| 21 | 25 | ) | |
| 22 | 26 | ||
| 23 | 27 | func runnerUsage() string { | |
| 24 | return `usage: vole runner --remote ssh://git@host[:port] [--once] [--dir path] | ||
| 28 | return `usage: vole runner --remote ssh://git@host[:port] [--once] [--dir path] [--key path] | ||
| 25 | 29 | ||
| 26 | 30 | Leases CI jobs from a Vole instance and runs them. Authentication is the SSH key | |
| 27 | 31 | you would push with, so there is no token to configure. A runner may only run jobs | |
| 28 | 32 | for repositories where its account has admin. | |
| 29 | 33 | ||
| 34 | Every finished attempt is signed with --key, because the runner is the party that | ||
| 35 | did the work: an unattributed result is what the queue already had. The key must | ||
| 36 | belong to the account the connection authenticated as, and is checked on connect | ||
| 37 | rather than after the build. | ||
| 38 | |||
| 30 | 39 | --once run one job, then exit (useful for a container that is scheduled per job) | |
| 31 | --dir where to keep checkouts (default: os.UserCacheDir()/vole-runner)` | ||
| 40 | --dir where to keep checkouts (default: os.UserCacheDir()/vole-runner) | ||
| 41 | --key private key to sign results with (default: -i from $VOLE_SSH_COMMAND or | ||
| 42 | $GIT_SSH_COMMAND, else ~/.ssh/id_ed25519)` | ||
| 32 | 43 | } | |
| 33 | 44 | ||
| 34 | 45 | type runnerStep struct { | |
| @@ -38,13 +49,19 @@ type runnerStep struct { | |||
| 38 | 49 | } | |
| 39 | 50 | ||
| 40 | 51 | type runnerJob struct { | |
| 41 | ID string `json:"id"` | ||
| 42 | Repo string `json:"repo"` | ||
| 43 | SHA string `json:"sha"` | ||
| 44 | Branch string `json:"branch"` | ||
| 45 | Workflow string `json:"workflow"` | ||
| 46 | Job string `json:"job"` | ||
| 47 | Steps []runnerStep `json:"steps"` | ||
| 52 | ID string `json:"id"` | ||
| 53 | Repo string `json:"repo"` | ||
| 54 | SHA string `json:"sha"` | ||
| 55 | // Ref, WorkflowSHA and StepsSHA are part of what this runner signs when the | ||
| 56 | // attempt ends: the exact revision, the exact workflow file, and the digest of | ||
| 57 | // the steps that were pinned when the job was queued. | ||
| 58 | Ref string `json:"ref"` | ||
| 59 | WorkflowSHA string `json:"workflow_sha"` | ||
| 60 | StepsSHA string `json:"steps_sha"` | ||
| 61 | Branch string `json:"branch"` | ||
| 62 | Workflow string `json:"workflow"` | ||
| 63 | Job string `json:"job"` | ||
| 64 | Steps []runnerStep `json:"steps"` | ||
| 48 | 65 | // Artifacts are path patterns, expanded in the checkout after the steps pass. | |
| 49 | 66 | Artifacts []string `json:"artifacts,omitempty"` | |
| 50 | 67 | // Tag is set for a tag-triggered run; TagAnnotated says whether it carries | |
| @@ -81,6 +98,11 @@ type runnerRequest struct { | |||
| 81 | 98 | LogSize int64 `json:"log_size,omitempty"` | |
| 82 | 99 | // Artifacts ride the finish so the result and the outputs are one message. | |
| 83 | 100 | Artifacts []runnerArtifact `json:"artifacts,omitempty"` | |
| 101 | // Finished is this runner's own unix time for the end of the attempt, and | ||
| 102 | // Signature covers the claim built from this message and the job: what was run, | ||
| 103 | // what came of it, and the objects it left behind. | ||
| 104 | Finished int64 `json:"finished,omitempty"` | ||
| 105 | Signature []byte `json:"signature,omitempty"` | ||
| 84 | 106 | } | |
| 85 | 107 | ||
| 86 | 108 | type runnerReply struct { | |
| @@ -89,6 +111,12 @@ type runnerReply struct { | |||
| 89 | 111 | Run *runnerJob `json:"run,omitempty"` | |
| 90 | 112 | OK bool `json:"ok"` | |
| 91 | 113 | Lease int `json:"lease,omitempty"` | |
| 114 | // The account this session authenticated as and the fingerprints of its keys, | ||
| 115 | // which is how a runner finds out before it starts work whether the key it will | ||
| 116 | // sign with is one the server will accept. | ||
| 117 | Account string `json:"account,omitempty"` | ||
| 118 | Name string `json:"name,omitempty"` | ||
| 119 | Keys []string `json:"keys,omitempty"` | ||
| 92 | 120 | } | |
| 93 | 121 | ||
| 94 | 122 | // cmdRunner leases and executes CI jobs. The protocol runs over an SSH channel | |
| @@ -96,7 +124,7 @@ type runnerReply struct { | |||
| 96 | 124 | // is: no separate credential, nothing stored. | |
| 97 | 125 | func cmdRunner(args []string) error { | |
| 98 | 126 | f, err := parseFlags(args, flagSpec{long: "remote", short: "r"}, flagSpec{long: "dir"}, | |
| 99 | flagSpec{long: "once", boolean: true}) | ||
| 127 | flagSpec{long: "key"}, flagSpec{long: "once", boolean: true}) | ||
| 100 | 128 | if err != nil { | |
| 101 | 129 | return err | |
| 102 | 130 | } | |
| @@ -126,7 +154,27 @@ func cmdRunner(args []string) error { | |||
| 126 | 154 | } | |
| 127 | 155 | defer sess.close() | |
| 128 | 156 | ||
| 129 | fmt.Fprintf(os.Stderr, "vole runner: connected to %s, waiting for work\n", remote) | ||
| 157 | key, fingerprint, err := loadSigningKey(f.one("key")) | ||
| 158 | if err != nil { | ||
| 159 | return err | ||
| 160 | } | ||
| 161 | sess.key = key | ||
| 162 | |||
| 163 | // Ask who the server thinks this is, and refuse to start work if the key we sign | ||
| 164 | // with is not one that account can answer for: learning that after a long build | ||
| 165 | // is a wasted build. | ||
| 166 | who, err := sess.call(runnerRequest{Op: "ping"}) | ||
| 167 | if err != nil { | ||
| 168 | return err | ||
| 169 | } | ||
| 170 | if who.Error != "" { | ||
| 171 | return errors.New(who.Error) | ||
| 172 | } | ||
| 173 | if err := checkSigningKey(who, fingerprint); err != nil { | ||
| 174 | return err | ||
| 175 | } | ||
| 176 | fmt.Fprintf(os.Stderr, "vole runner: connected as %s, signing with %s; waiting for work\n", | ||
| 177 | who.Name, fingerprint) | ||
| 130 | 178 | for { | |
| 131 | 179 | rep, err := sess.call(runnerRequest{Op: "lease"}) | |
| 132 | 180 | if err != nil { | |
| @@ -162,6 +210,83 @@ type runnerSession struct { | |||
| 162 | 210 | enc *json.Encoder | |
| 163 | 211 | dec *json.Decoder | |
| 164 | 212 | mu sync.Mutex | |
| 213 | // key signs every finish this session sends. It is the runner's own key, not an | ||
| 214 | // instance secret: the record names the party that did the work. | ||
| 215 | key ed25519.PrivateKey | ||
| 216 | } | ||
| 217 | |||
| 218 | // checkSigningKey is the refusal that has to happen before work, not after it: the | ||
| 219 | // server verifies a finish against the keys of the account the connection | ||
| 220 | // authenticated as, so a runner holding any other key would run a job it cannot then | ||
| 221 | // report. | ||
| 222 | func checkSigningKey(who runnerReply, fingerprint string) error { | ||
| 223 | if slices.Contains(who.Keys, fingerprint) { | ||
| 224 | return nil | ||
| 225 | } | ||
| 226 | return fmt.Errorf("this runner signs with %s, which is not a key of %s; "+ | ||
| 227 | "pass --key with a key that account holds (it has %d key(s))", fingerprint, who.Name, len(who.Keys)) | ||
| 228 | } | ||
| 229 | |||
| 230 | // loadSigningKey finds and reads the key a runner signs with: --key, then the -i of | ||
| 231 | // the ssh command the session itself is started with, then the usual default. An | ||
| 232 | // agent holds keys but does not sign arbitrary bytes, so a CI runner keeps its key | ||
| 233 | // on disk — which is the one place it already has it. | ||
| 234 | func loadSigningKey(flag string) (ed25519.PrivateKey, string, error) { | ||
| 235 | path := flag | ||
| 236 | if path == "" { | ||
| 237 | path = keyFromSSHCommand() | ||
| 238 | } | ||
| 239 | if path == "" { | ||
| 240 | if home, err := os.UserHomeDir(); err == nil { | ||
| 241 | cand := filepath.Join(home, ".ssh", "id_ed25519") | ||
| 242 | if _, err := os.Stat(cand); err == nil { | ||
| 243 | path = cand | ||
| 244 | } | ||
| 245 | } | ||
| 246 | } | ||
| 247 | if path == "" { | ||
| 248 | return nil, "", errors.New("no signing key: pass --key with the private key of the account this runner connects as") | ||
| 249 | } | ||
| 250 | data, err := os.ReadFile(path) | ||
| 251 | if err != nil { | ||
| 252 | return nil, "", fmt.Errorf("reading the signing key: %w", err) | ||
| 253 | } | ||
| 254 | raw, err := ssh.ParseRawPrivateKey(data) | ||
| 255 | if err != nil { | ||
| 256 | return nil, "", fmt.Errorf("%s: %w (a runner needs an unencrypted key: nothing is there to type a passphrase)", path, err) | ||
| 257 | } | ||
| 258 | var key ed25519.PrivateKey | ||
| 259 | switch k := raw.(type) { | ||
| 260 | case *ed25519.PrivateKey: | ||
| 261 | key = *k | ||
| 262 | case ed25519.PrivateKey: | ||
| 263 | key = k | ||
| 264 | default: | ||
| 265 | return nil, "", fmt.Errorf("%s is a %T key; a runner signs with ed25519", path, raw) | ||
| 266 | } | ||
| 267 | pub, err := ssh.NewPublicKey(key.Public()) | ||
| 268 | if err != nil { | ||
| 269 | return nil, "", err | ||
| 270 | } | ||
| 271 | return key, ssh.FingerprintSHA256(pub), nil | ||
| 272 | } | ||
| 273 | |||
| 274 | // keyFromSSHCommand reads -i out of the ssh command this runner is started with, so | ||
| 275 | // `VOLE_SSH_COMMAND="ssh -i /path/ci" vole runner` signs with the same key it | ||
| 276 | // connects with instead of making the operator say it twice. | ||
| 277 | func keyFromSSHCommand() string { | ||
| 278 | for _, env := range []string{"VOLE_SSH_COMMAND", "GIT_SSH_COMMAND"} { | ||
| 279 | fields := strings.Fields(os.Getenv(env)) | ||
| 280 | for i, a := range fields { | ||
| 281 | if a == "-i" && i+1 < len(fields) { | ||
| 282 | return fields[i+1] | ||
| 283 | } | ||
| 284 | if strings.HasPrefix(a, "-i") && len(a) > 2 { | ||
| 285 | return a[2:] | ||
| 286 | } | ||
| 287 | } | ||
| 288 | } | ||
| 289 | return "" | ||
| 165 | 290 | } | |
| 166 | 291 | ||
| 167 | 292 | func dialRunner(remote string) (*runnerSession, error) { | |
| @@ -285,20 +410,48 @@ func (s *runnerSession) run(remote, cache string, job *runnerJob, lease time.Dur | |||
| 285 | 410 | s.finish(remote, job, log, 0, arts) | |
| 286 | 411 | } | |
| 287 | 412 | ||
| 288 | // finish uploads this attempt's log and then reports the outcome. The bytes go in | ||
| 289 | // first, the same rule artifacts follow: the server refuses a log it cannot serve, | ||
| 290 | // and the object id is one this runner computed from the content. | ||
| 413 | // finish uploads this attempt's log, signs the result and reports it. | ||
| 291 | 414 | // | |
| 292 | // A log that cannot be stored does not lose the result: the run is finished without | ||
| 293 | // one, and the failure is reported on stderr rather than swallowed. | ||
| 415 | // The bytes go in first, the same rule artifacts follow: the server refuses a log it | ||
| 416 | // cannot serve, and the object id is one this runner computed from the content — so | ||
| 417 | // the log is a fact the signature can cover. | ||
| 418 | // | ||
| 419 | // A log that cannot be stored does not lose the result: the claim then names no log | ||
| 420 | // and the failure is reported on stderr rather than swallowed. A claim that cannot be | ||
| 421 | // signed is not sent at all, because the server would refuse it. | ||
| 294 | 422 | func (s *runnerSession) finish(remote string, job *runnerJob, log *strings.Builder, exit int, arts []runnerArtifact) { | |
| 295 | 423 | oid, size, err := uploadLog(remote, job.Repo, log.String()) | |
| 296 | 424 | if err != nil { | |
| 297 | 425 | fmt.Fprintf(os.Stderr, "vole runner: could not upload the log for %s: %v\n", job.ID, err) | |
| 298 | 426 | } | |
| 427 | artifacts := make([]runs.Artifact, 0, len(arts)) | ||
| 428 | for _, a := range arts { | ||
| 429 | artifacts = append(artifacts, runs.Artifact{Name: a.Name, OID: a.OID, Size: a.Size}) | ||
| 430 | } | ||
| 431 | now := time.Now().UTC() | ||
| 432 | payload, err := runs.Claim{ | ||
| 433 | RunID: job.ID, | ||
| 434 | Repo: job.Repo, | ||
| 435 | Ref: job.Ref, | ||
| 436 | SHA: job.SHA, | ||
| 437 | Workflow: job.Workflow, | ||
| 438 | WorkflowSHA: job.WorkflowSHA, | ||
| 439 | Job: job.Job, | ||
| 440 | StepsSHA: job.StepsSHA, | ||
| 441 | Exit: exit, | ||
| 442 | LogOID: oid, | ||
| 443 | LogSize: size, | ||
| 444 | Artifacts: artifacts, | ||
| 445 | Finished: now.Unix(), | ||
| 446 | }.Canonical() | ||
| 447 | if err != nil { | ||
| 448 | fmt.Fprintf(os.Stderr, "vole runner: cannot describe the result of %s: %v\n", job.ID, err) | ||
| 449 | return | ||
| 450 | } | ||
| 299 | 451 | _, _ = s.call(runnerRequest{ | |
| 300 | 452 | Op: "finish", Run: job.ID, Exit: exit, | |
| 301 | 453 | LogOID: oid, LogSize: size, Artifacts: arts, | |
| 454 | Finished: now.Unix(), Signature: ed25519.Sign(s.key, payload), | ||
| 302 | 455 | }) | |
| 303 | 456 | } | |
| 304 | 457 | ||