vole
Clone
Read only, no credentials:
https://rodent.software/vole.git
Write over SSH with a registered key:
git clone ssh://git@rodent.software:2222/vole.git
A free and open-source forge that lives in your Git repo.
Phase 4: the runner signs the result it reports
22528ac3da576387a8311969641a7b5783b04924 · Keenan Nemetz · 17d ago · parent 86fe5bb4
Phase 4: the runner signs the result it reports
An instance-key signature is the server witnessing itself. It cannot be checked
without trusting the box that produced it, and it says nothing about the party that
ran the steps — so a record signed that way would have been paperwork. The runner
already holds a key, the one it connects with, so the runner is what signs.
At finish the runner signs a claim: run, repo, ref, SHA, workflow blob, the digest of
the steps pinned when the job was queued, exit code, the log's {oid, size}, the
artifacts it published, and its own finishing time. The server rebuilds the same claim
from its own row plus what the runner reports, verifies the signature, and stores the
payload and signature verbatim — so a verifier re-checks the bytes that were signed
rather than re-deriving them from a format that might have drifted.
Who the signer is comes from the channel, not the claim: the account is the one the
SSH key resolved to, and the signature must verify under one of that account's keys.
A runner may therefore sign with a different key from the one it connected with — an
agent-held key to connect, a file to sign — but not as somebody else.
Refusals, all before anything is recorded, because half a record is worse than none:
an unsigned finish (an unattributed result is what the queue already had), a signature
by another account's key, and a signature over a different claim. The run page shows
`attested by <name> · key <fingerprint>`, and `vole runner` now pings first and refuses
to start work when the key it will sign with is not one its account holds — discovering
that after a ten-minute build is a wasted build.
`steps_sha` is recorded at enqueue and travels with the lease, so the claim covers the
steps that ran and not merely the workflow file they came from.
What this does not buy: truth. A runner can still lie about an exit code. It buys
accountability — the record is attributable to a party other than the instance.
Whether a required check must be attested by a particular runner (an ACL key defaulting
to the instance's own) is the next policy question and is deliberately not answered
here; any admin of the repository may still supply the signature.
Verified: TestAFinishedRunBelongsToTheRunnerThatSignsIt drives the real protocol over
SSH — the lease carries what the claim names, ping names the account and its keys,
unsigned and wrong-account and wrong-claim finishes are all refused with nothing
recorded, the correct one is accepted, and the stored payload still verifies under the
stored signature; TestClaimRendersOneWay pins the byte form (including artifact
ordering and an empty log) and TestClaimRefusesToForgeItself refuses a field carrying a
newline; TestStepDigestFollowsTheSteps shows the digest covering the steps themselves;
the cmd tests cover key discovery from VOLE_SSH_COMMAND/GIT_SSH_COMMAND, the missing-key
message, a non-key file, and the pre-work refusal; and the end-to-end test that runs the
real binary asserts the claim names the run, its log and its exit, and that the run page
says who attested it. scripts/ci.sh green (gofmt, vet, build, markdown links, go test
-race ./...).
internal/sshserver/runner.go +151−28
| @@ -1,6 +1,7 @@ | |||
| 1 | 1 | package sshserver | |
| 2 | 2 | ||
| 3 | 3 | import ( | |
| 4 | "crypto/ed25519" | ||
| 4 | 5 | "encoding/json" | |
| 5 | 6 | "fmt" | |
| 6 | 7 | "time" | |
| @@ -49,6 +50,12 @@ type runnerRequest struct { | |||
| 49 | 50 | // and its result are one fact, so there is no window in which a successful run | |
| 50 | 51 | // has no artifacts or a failed one has some. | |
| 51 | 52 | Artifacts []runnerArtifact `json:"artifacts,omitempty"` | |
| 53 | // Finished is the runner's own unix time for when the attempt ended, and | ||
| 54 | // Signature is its signature over the claim that includes it. A finish without | ||
| 55 | // a signature is refused: an unattributed result is what the queue already had, | ||
| 56 | // and the runner is the only party that can be held to it. | ||
| 57 | Finished int64 `json:"finished,omitempty"` | ||
| 58 | Signature []byte `json:"signature,omitempty"` | ||
| 52 | 59 | } | |
| 53 | 60 | ||
| 54 | 61 | type runnerArtifact struct { | |
| @@ -64,13 +71,20 @@ type runnerStep struct { | |||
| 64 | 71 | } | |
| 65 | 72 | ||
| 66 | 73 | type runnerJob struct { | |
| 67 | ID string `json:"id"` | ||
| 68 | Repo string `json:"repo"` | ||
| 69 | SHA string `json:"sha"` | ||
| 70 | Branch string `json:"branch"` | ||
| 71 | Workflow string `json:"workflow"` | ||
| 72 | Job string `json:"job"` | ||
| 73 | Steps []runnerStep `json:"steps"` | ||
| 74 | ID string `json:"id"` | ||
| 75 | Repo string `json:"repo"` | ||
| 76 | SHA string `json:"sha"` | ||
| 77 | // Ref, WorkflowSHA and StepsSHA are what the runner signs when the attempt ends: | ||
| 78 | // the exact revision, the exact workflow file, and the digest of the steps that | ||
| 79 | // were pinned when the job was queued. Without them the claim could only say | ||
| 80 | // "some job", which is not a statement anybody can check later. | ||
| 81 | Ref string `json:"ref"` | ||
| 82 | WorkflowSHA string `json:"workflow_sha"` | ||
| 83 | StepsSHA string `json:"steps_sha"` | ||
| 84 | Branch string `json:"branch"` | ||
| 85 | Workflow string `json:"workflow"` | ||
| 86 | Job string `json:"job"` | ||
| 87 | Steps []runnerStep `json:"steps"` | ||
| 74 | 88 | // Artifacts are the declared patterns the runner expands and uploads after the | |
| 75 | 89 | // steps succeed. | |
| 76 | 90 | Artifacts []string `json:"artifacts,omitempty"` | |
| @@ -97,6 +111,13 @@ type runnerReply struct { | |||
| 97 | 111 | Run *runnerJob `json:"run,omitempty"` | |
| 98 | 112 | OK bool `json:"ok"` | |
| 99 | 113 | Lease int `json:"lease,omitempty"` | |
| 114 | // Who am I: the account this channel authenticated as, and the fingerprints of | ||
| 115 | // its keys. A runner answers "ping" with these so it can check, before it runs | ||
| 116 | // anything, that the key it is going to sign with is one the server will accept. | ||
| 117 | // Learning that after a ten-minute build is too late to be useful. | ||
| 118 | Account string `json:"account,omitempty"` | ||
| 119 | Name string `json:"name,omitempty"` | ||
| 120 | Keys []string `json:"keys,omitempty"` | ||
| 100 | 121 | } | |
| 101 | 122 | ||
| 102 | 123 | // serveRunner answers one runner session until it disconnects. | |
| @@ -138,7 +159,8 @@ func (s *Server) serveRunner(ch ssh.Channel, user, accountID string) { | |||
| 138 | 159 | } | |
| 139 | 160 | job := &runnerJob{ | |
| 140 | 161 | ID: run.ID, Repo: run.Repo, SHA: run.SHA, Branch: run.Branch, | |
| 141 | Workflow: run.Workflow, Job: run.Job, Artifacts: run.Artifacts, | ||
| 162 | Ref: run.Ref, Workflow: run.Workflow, WorkflowSHA: run.WorkflowSHA, | ||
| 163 | StepsSHA: run.StepsSHA, Job: run.Job, Artifacts: run.Artifacts, | ||
| 142 | 164 | Tag: run.Tag, TagAnnotated: run.TagAnnotated, ReleaseArtifacts: run.ReleaseArtifacts, | |
| 143 | 165 | } | |
| 144 | 166 | if run.PagesPath != "" { | |
| @@ -165,15 +187,44 @@ func (s *Server) serveRunner(ch ssh.Channel, user, accountID string) { | |||
| 165 | 187 | _ = enc.Encode(runnerReply{Op: "heartbeat", OK: ok}) | |
| 166 | 188 | ||
| 167 | 189 | case "finish": | |
| 168 | // Artifacts are recorded while the lease is still held, and only if the | ||
| 169 | // objects are really in the store: a runner naming bytes it never | ||
| 170 | // uploaded would otherwise produce a download that 404s. | ||
| 171 | if len(req.Artifacts) > 0 { | ||
| 172 | arts, err := s.checkArtifacts(req.Artifacts) | ||
| 173 | if err != nil { | ||
| 190 | // Everything is checked before anything is recorded. A finish that | ||
| 191 | // cannot be attributed must not leave artifacts, a log or a result | ||
| 192 | // behind: the point of the signature is that the record names the party | ||
| 193 | // that produced it, and half a record is worse than none. | ||
| 194 | // | ||
| 195 | // Artifacts and the log are checked against the store — a runner naming | ||
| 196 | // bytes it never uploaded would otherwise produce a download that 404s — | ||
| 197 | // and the claim is assembled from the row plus what the runner says it | ||
| 198 | // produced, then verified under the account the channel authenticated as. | ||
| 199 | arts, err := s.checkArtifacts(req.Artifacts) | ||
| 200 | if err != nil { | ||
| 201 | _ = enc.Encode(runnerReply{Op: "finish", Error: err.Error()}) | ||
| 202 | continue | ||
| 203 | } | ||
| 204 | if req.LogOID != "" { | ||
| 205 | if err := s.checkRunLog(req.LogOID, req.LogSize); err != nil { | ||
| 174 | 206 | _ = enc.Encode(runnerReply{Op: "finish", Error: err.Error()}) | |
| 175 | 207 | continue | |
| 176 | 208 | } | |
| 209 | } | ||
| 210 | payload, err := s.claimFor(req, arts) | ||
| 211 | if err != nil { | ||
| 212 | _ = enc.Encode(runnerReply{Op: "finish", Error: err.Error()}) | ||
| 213 | continue | ||
| 214 | } | ||
| 215 | if len(req.Signature) == 0 { | ||
| 216 | _ = enc.Encode(runnerReply{Op: "finish", | ||
| 217 | Error: "a finish must be signed by the runner that ran the job: run `vole runner --key <private key>`"}) | ||
| 218 | continue | ||
| 219 | } | ||
| 220 | fingerprint, err := s.verifyClaim(accountID, payload, req.Signature) | ||
| 221 | if err != nil { | ||
| 222 | _ = enc.Encode(runnerReply{Op: "finish", Error: err.Error()}) | ||
| 223 | continue | ||
| 224 | } | ||
| 225 | |||
| 226 | now := time.Now().UTC() | ||
| 227 | if len(arts) > 0 { | ||
| 177 | 228 | ok, err := cfg.Runs.RecordArtifacts(req.Run, owner, arts) | |
| 178 | 229 | if err != nil { | |
| 179 | 230 | _ = enc.Encode(runnerReply{Op: "finish", Error: err.Error()}) | |
| @@ -185,37 +236,49 @@ func (s *Server) serveRunner(ch ssh.Channel, user, accountID string) { | |||
| 185 | 236 | continue | |
| 186 | 237 | } | |
| 187 | 238 | } | |
| 188 | // The log is the runner's own bytes, uploaded to the content-addressed | ||
| 189 | // store before this message: the store is the authority on whether they | ||
| 190 | // exist, exactly as for artifacts, and the object id was computed by the | ||
| 191 | // party that wrote them. The server never assembles a log, which is what | ||
| 192 | // makes the archived bytes something a runner can attest to. | ||
| 193 | 239 | if req.LogOID != "" { | |
| 194 | if err := s.checkRunLog(req.LogOID, req.LogSize); err != nil { | ||
| 195 | _ = enc.Encode(runnerReply{Op: "finish", Error: err.Error()}) | ||
| 196 | continue | ||
| 197 | } | ||
| 198 | recorded, err := cfg.Runs.RecordLog(req.Run, owner, req.LogOID, req.LogSize, time.Now().UTC()) | ||
| 240 | recorded, err := cfg.Runs.RecordLog(req.Run, owner, req.LogOID, req.LogSize, now) | ||
| 199 | 241 | if err != nil { | |
| 200 | 242 | _ = enc.Encode(runnerReply{Op: "finish", Error: err.Error()}) | |
| 201 | 243 | continue | |
| 202 | 244 | } | |
| 203 | 245 | if !recorded { | |
| 204 | // The lease moved: report nothing, like a lost finish. | ||
| 205 | 246 | _ = enc.Encode(runnerReply{Op: "finish", OK: false}) | |
| 206 | 247 | continue | |
| 207 | 248 | } | |
| 208 | 249 | } | |
| 209 | ok, err := cfg.Runs.Finish(req.Run, owner, req.Exit, time.Now().UTC()) | ||
| 250 | attested, err := cfg.Runs.RecordAttestation(req.Run, owner, runs.Attestation{ | ||
| 251 | Signer: accountID, | ||
| 252 | Fingerprint: fingerprint, | ||
| 253 | Payload: payload, | ||
| 254 | Signature: req.Signature, | ||
| 255 | }, now) | ||
| 210 | 256 | if err != nil { | |
| 211 | 257 | _ = enc.Encode(runnerReply{Op: "finish", Error: err.Error()}) | |
| 212 | 258 | continue | |
| 213 | 259 | } | |
| 214 | cfg.Log.Info("run finished", "user", user, "run", req.Run, "exit", req.Exit, "recorded", ok) | ||
| 260 | if !attested { | ||
| 261 | _ = enc.Encode(runnerReply{Op: "finish", OK: false}) | ||
| 262 | continue | ||
| 263 | } | ||
| 264 | ok, err := cfg.Runs.Finish(req.Run, owner, req.Exit, now) | ||
| 265 | if err != nil { | ||
| 266 | _ = enc.Encode(runnerReply{Op: "finish", Error: err.Error()}) | ||
| 267 | continue | ||
| 268 | } | ||
| 269 | cfg.Log.Info("run finished", "user", user, "run", req.Run, "exit", req.Exit, | ||
| 270 | "signer", accountID, "key", fingerprint, "recorded", ok) | ||
| 215 | 271 | _ = enc.Encode(runnerReply{Op: "finish", OK: ok}) | |
| 216 | 272 | ||
| 217 | 273 | case "ping": | |
| 218 | _ = enc.Encode(runnerReply{Op: "ping", OK: true}) | ||
| 274 | reply := runnerReply{Op: "ping", OK: true, Account: accountID} | ||
| 275 | if acct, aerr := cfg.Identity.Find(accountID); aerr == nil && acct != nil { | ||
| 276 | reply.Name = acct.Name | ||
| 277 | for _, k := range acct.Keys { | ||
| 278 | reply.Keys = append(reply.Keys, k.Fingerprint) | ||
| 279 | } | ||
| 280 | } | ||
| 281 | _ = enc.Encode(reply) | ||
| 219 | 282 | ||
| 220 | 283 | default: | |
| 221 | 284 | _ = enc.Encode(runnerReply{Op: req.Op, Error: "unknown op"}) | |
| @@ -226,6 +289,66 @@ func (s *Server) serveRunner(ch ssh.Channel, user, accountID string) { | |||
| 226 | 289 | // checkArtifacts validates what a runner claims it uploaded. The store is the | |
| 227 | 290 | // authority on whether the bytes exist; this only refuses claims that are not | |
| 228 | 291 | // downloadable at all. | |
| 292 | // claimFor assembles the statement a runner signs: the job as the server recorded | ||
| 293 | // it, plus what the runner reports it produced. Both sides build it from one value, | ||
| 294 | // so a disagreement in either direction is a claim that does not verify. | ||
| 295 | func (s *Server) claimFor(req runnerRequest, arts []runs.Artifact) ([]byte, error) { | ||
| 296 | run, err := s.cfg.Runs.Get(req.Run) | ||
| 297 | if err != nil { | ||
| 298 | return nil, fmt.Errorf("reading run %s: %w", req.Run, err) | ||
| 299 | } | ||
| 300 | if run == nil { | ||
| 301 | return nil, fmt.Errorf("no run %s", req.Run) | ||
| 302 | } | ||
| 303 | return runs.Claim{ | ||
| 304 | RunID: run.ID, | ||
| 305 | Repo: run.Repo, | ||
| 306 | Ref: run.Ref, | ||
| 307 | SHA: run.SHA, | ||
| 308 | Workflow: run.Workflow, | ||
| 309 | WorkflowSHA: run.WorkflowSHA, | ||
| 310 | Job: run.Job, | ||
| 311 | StepsSHA: run.StepsSHA, | ||
| 312 | Exit: req.Exit, | ||
| 313 | LogOID: req.LogOID, | ||
| 314 | LogSize: req.LogSize, | ||
| 315 | Artifacts: arts, | ||
| 316 | Finished: req.Finished, | ||
| 317 | }.Canonical() | ||
| 318 | } | ||
| 319 | |||
| 320 | // verifyClaim checks a signature over the claim against the keys of the account the | ||
| 321 | // channel authenticated as, and returns the fingerprint that matched. | ||
| 322 | // | ||
| 323 | // The account is not the claim's to choose: it is the identity the server resolved | ||
| 324 | // from the SSH key that opened the connection. The signing key may differ from the | ||
| 325 | // connecting one — an agent-held key to connect, a file to sign — but it must belong | ||
| 326 | // to the same account, or a runner could sign as somebody else. | ||
| 327 | func (s *Server) verifyClaim(accountID string, payload, sig []byte) (string, error) { | ||
| 328 | acct, err := s.cfg.Identity.Find(accountID) | ||
| 329 | if err != nil || acct == nil { | ||
| 330 | return "", fmt.Errorf("cannot read the account %s: %v", accountID, err) | ||
| 331 | } | ||
| 332 | for _, k := range acct.Keys { | ||
| 333 | pub, _, _, _, err := ssh.ParseAuthorizedKey([]byte(k.Line)) | ||
| 334 | if err != nil { | ||
| 335 | continue | ||
| 336 | } | ||
| 337 | cpk, ok := pub.(ssh.CryptoPublicKey) | ||
| 338 | if !ok { | ||
| 339 | continue | ||
| 340 | } | ||
| 341 | ed, ok := cpk.CryptoPublicKey().(ed25519.PublicKey) | ||
| 342 | if !ok { | ||
| 343 | continue | ||
| 344 | } | ||
| 345 | if ed25519.Verify(ed, payload, sig) { | ||
| 346 | return k.Fingerprint, nil | ||
| 347 | } | ||
| 348 | } | ||
| 349 | return "", fmt.Errorf("the signature does not verify under any key of %s: sign with the key you push with", acct.Name) | ||
| 350 | } | ||
| 351 | |||
| 229 | 352 | // checkRunLog verifies a log claim the way checkArtifacts verifies an artifact | |
| 230 | 353 | // claim: the bytes must be in the store, and the declared size must be theirs. | |
| 231 | 354 | func (s *Server) checkRunLog(oid string, size int64) error { | |