vole
Clone
Read only, no credentials:
https://rodent.software/git/vole.git
Write over SSH with a registered key:
git clone ssh://git@0.0.0.0:2222/vole.git
A free and open-source forge that lives in your Git repo.
Stop a running job at its runner's next heartbeat
0c14f0789af1ab0c43f99043acd7507eec4956e4 · Keenan Nemetz · 6d ago · parent 7e0b5950
Stop a running job at its runner's next heartbeat A maintainer could cancel a queued run and nothing else: CancelRun only retired waiting work, so a build that was clearly wrong or expensive held its runner until its budget ran out, or until somebody killed the container by hand. The queue cannot kill a process it does not own, but it can ask. Canceling a running run now marks the queue row and leaves the lease alone, because the runner still holds it and its partial attempt has to be acceptable. The runner sees the request on the heartbeat it already sends while a step or an image build runs, stops the container, writes one line to the log, and reports what it did. RecordAttempt and Finish then keep the canceled status instead of mapping the nonzero exit code to failed, so the partial log and step timings survive and the run is not a red build. A stop request whose runner dies first becomes canceled on the expiry sweep rather than being leased again, and a rerun clears the request: it is a new decision. The run page says who asked, distinguishes a run stopped while running from one canceled in the queue, and shows the interrupted step as stopped rather than failed. Closes #162.
server/ssh/runner.go +10−3
| @@ -243,6 +243,10 @@ type runnerReply struct { | |||
| 243 | 243 | Repos []string `json:"repos,omitempty"` | |
| 244 | 244 | // LogSize is how much of the log the server has after a log chunk. | |
| 245 | 245 | LogSize int64 `json:"log_size,omitempty"` | |
| 246 | // Cancel names the account that asked this running job to stop. It rides the | ||
| 247 | // heartbeat reply, so stopping a run needs no new traffic and a runner too old to | ||
| 248 | // know the field ignores it. Empty means nobody has asked. | ||
| 249 | Cancel string `json:"cancel,omitempty"` | ||
| 246 | 250 | } | |
| 247 | 251 | ||
| 248 | 252 | // serveRunner answers one runner session until it disconnects. | |
| @@ -437,14 +441,17 @@ func (s *Server) serveRunner(ch xssh.Channel, user string) { | |||
| 437 | 441 | ||
| 438 | 442 | case "heartbeat": | |
| 439 | 443 | // Liveness only: the lease is what a heartbeat buys. Presence is refreshed | |
| 440 | // here too, so a long build does not make its runner look gone. | ||
| 444 | // here too, so a long build does not make its runner look gone. A stop request | ||
| 445 | // rides the same reply rather than a new message: the runner already asks this | ||
| 446 | // question every few seconds, and a runner that does not know the field keeps | ||
| 447 | // working exactly as before. | ||
| 441 | 448 | _ = cfg.Runs.TouchSession(sessionID, user, nil, time.Now().UTC()) | |
| 442 | ok, err := cfg.Runs.Heartbeat(req.Run, owner, cfg.Lease, time.Now().UTC()) | ||
| 449 | held, cancelBy, err := cfg.Runs.Heartbeat(req.Run, owner, cfg.Lease, time.Now().UTC()) | ||
| 443 | 450 | if err != nil { | |
| 444 | 451 | _ = enc.Encode(runnerReply{Op: "heartbeat", Error: err.Error()}) | |
| 445 | 452 | continue | |
| 446 | 453 | } | |
| 447 | _ = enc.Encode(runnerReply{Op: "heartbeat", OK: ok}) | ||
| 454 | _ = enc.Encode(runnerReply{Op: "heartbeat", OK: held, Cancel: cancelBy}) | ||
| 448 | 455 | ||
| 449 | 456 | case "finish": | |
| 450 | 457 | // The result arrives here, not on the metadata branch. The record is | |