vole
Clone
Read only, no credentials:
https://rodent.software/git/vole.git
Write over SSH with a registered key:
git clone ssh://git@0.0.0.0:2222/vole.git
A free and open-source forge that lives in your Git repo.
Phase 4 slice 2: the queue and the runner
b44cbfaf873fda01a458b527e5684b2386266117 · Keenan Nemetz · 17d ago · parent 8fe238d4
Phase 4 slice 2: the queue and the runner A push now runs a pipeline. post-receive spools one file per accepted ref update — the hook must never write to a database or fail a commit — and the server ingests those into data/runs.db, pinning each run to the commit and to the workflow blob it was queued from. The unique index on (repo, ref, sha, workflow, job) is what makes ingestion idempotent, so a spool file read twice after a crash does not run twice. vole runner connects over SSH with the key it would push with: no token, no stored secret, and the same identity the rest of the system uses. It may only run jobs for repositories where its account is an admin, because running a job executes that repository's code — a bigger power than writing to it. Delivery is at-least-once by construction: - A lease hands out a job for a fixed window; the runner heartbeats while it works. - An expired lease means the runner died, and the janitor returns the job to the queue for somebody else. - A runner whose lease is gone stops rather than reporting a result nobody will record, and a job that kills five runners in a row is failed instead of cycling. Three bugs found while building it, each worth the test that now covers it: - Background loops lived in Run(), so the tests — which serve the SSH listener themselves — never had ingestion or lease expiry. They start in New() now. - post-receive derived the repository name from GIT_DIR, which git sets to "." for hooks, so every event was queued for a repository called ".". The same trap had already appeared in a pre-receive message; both now resolve the absolute path. - Leases ordered only by creation time, so two pushes in the same second could be handed out in any order; the id breaks the tie, and ULIDs sort by time. Verified end to end on a live instance: a push queues a job, a runner leases it, clones the commit, prints "hi", and the run closes success with the log attached. The integration tests cover the two exit criteria that matter — a push runs a pipeline, and killing a runner mid-job requeues the work onto a second runner (attempts=2, success) — plus the property that editing a workflow after a push does not change what the already-queued run executes. Not built yet: statuses on change requests and required-check gating (slice 3), and signed run records (Phase 7). Runs live in data/runs.db and are visible there, not on the web.
internal/sshserver/sshserver.go +17
| @@ -28,6 +28,7 @@ import ( | |||
| 28 | 28 | ||
| 29 | 29 | "github.com/rodent-software/vole/internal/identity" | |
| 30 | 30 | "github.com/rodent-software/vole/internal/repo" | |
| 31 | "github.com/rodent-software/vole/internal/runs" | ||
| 31 | 32 | ) | |
| 32 | 33 | ||
| 33 | 34 | const serverVersion = "SSH-2.0-vole" | |
| @@ -40,6 +41,12 @@ type Config struct { | |||
| 40 | 41 | MaxBlobBytes int64 | |
| 41 | 42 | // AuditPath is handed to hooks so privileged operations can be recorded. | |
| 42 | 43 | AuditPath string | |
| 44 | // SpoolDir is where post-receive drops accepted push events. | ||
| 45 | SpoolDir string | ||
| 46 | // Runs is the CI queue, served to runners over the vole-runner verb. | ||
| 47 | Runs *runs.Store | ||
| 48 | // Lease is how long a runner may hold a job without a heartbeat. | ||
| 49 | Lease time.Duration | ||
| 43 | 50 | } | |
| 44 | 51 | ||
| 45 | 52 | type Server struct { | |
| @@ -182,6 +189,15 @@ func (s *Server) exec(ch ssh.Channel, req *ssh.Request, command, user, accountID | |||
| 182 | 189 | refuse("malformed command") | |
| 183 | 190 | return | |
| 184 | 191 | } | |
| 192 | // The CI runner is the one non-git verb. It reuses the same key identity rather | ||
| 193 | // than introducing a token, and it is a stream: the runner leases jobs and | ||
| 194 | // reports results over this channel. | ||
| 195 | if len(argv) == 1 && argv[0] == "vole-runner" { | ||
| 196 | req.Reply(true, nil) | ||
| 197 | s.serveRunner(ch, user, accountID) | ||
| 198 | exit(ch, 0) | ||
| 199 | return | ||
| 200 | } | ||
| 185 | 201 | if len(argv) != 2 { | |
| 186 | 202 | refuse("only git-upload-pack and git-receive-pack are available") | |
| 187 | 203 | return | |
| @@ -251,6 +267,7 @@ func (s *Server) exec(ch ssh.Channel, req *ssh.Request, command, user, accountID | |||
| 251 | 267 | "VOLE_MAX_BLOB="+strconv.FormatInt(s.cfg.MaxBlobBytes, 10), | |
| 252 | 268 | "VOLE_AUDIT="+s.cfg.AuditPath, | |
| 253 | 269 | "VOLE_SYSTEM_REPO="+s.cfg.Identity.Dir(), | |
| 270 | "VOLE_SPOOL="+s.cfg.SpoolDir, | ||
| 254 | 271 | ) | |
| 255 | 272 | cmd.Stdin = struct{ io.Reader }{ch} | |
| 256 | 273 | cmd.Stdout = struct{ io.Writer }{ch} | |