vole
Clone
Read only, no credentials:
https://rodent.software/git/vole.git
Write over SSH with a registered key:
git clone ssh://git@0.0.0.0:2222/vole.git
A free and open-source forge that lives in your Git repo.
Phase 4 slice 2: the queue and the runner
b44cbfaf873fda01a458b527e5684b2386266117 · Keenan Nemetz · 17d ago · parent 8fe238d4
Phase 4 slice 2: the queue and the runner A push now runs a pipeline. post-receive spools one file per accepted ref update — the hook must never write to a database or fail a commit — and the server ingests those into data/runs.db, pinning each run to the commit and to the workflow blob it was queued from. The unique index on (repo, ref, sha, workflow, job) is what makes ingestion idempotent, so a spool file read twice after a crash does not run twice. vole runner connects over SSH with the key it would push with: no token, no stored secret, and the same identity the rest of the system uses. It may only run jobs for repositories where its account is an admin, because running a job executes that repository's code — a bigger power than writing to it. Delivery is at-least-once by construction: - A lease hands out a job for a fixed window; the runner heartbeats while it works. - An expired lease means the runner died, and the janitor returns the job to the queue for somebody else. - A runner whose lease is gone stops rather than reporting a result nobody will record, and a job that kills five runners in a row is failed instead of cycling. Three bugs found while building it, each worth the test that now covers it: - Background loops lived in Run(), so the tests — which serve the SSH listener themselves — never had ingestion or lease expiry. They start in New() now. - post-receive derived the repository name from GIT_DIR, which git sets to "." for hooks, so every event was queued for a repository called ".". The same trap had already appeared in a pre-receive message; both now resolve the absolute path. - Leases ordered only by creation time, so two pushes in the same second could be handed out in any order; the id breaks the tie, and ULIDs sort by time. Verified end to end on a live instance: a push queues a job, a runner leases it, clones the commit, prints "hi", and the run closes success with the log attached. The integration tests cover the two exit criteria that matter — a push runs a pipeline, and killing a runner mid-job requeues the work onto a second runner (attempts=2, success) — plus the property that editing a workflow after a push does not change what the already-queued run executes. Not built yet: statuses on change requests and required-check gating (slice 3), and signed run records (Phase 7). Runs live in data/runs.db and are visible there, not on the web.
internal/workflow/workflow.go +34−1
| @@ -55,7 +55,10 @@ type Job struct { | |||
| 55 | 55 | } | |
| 56 | 56 | ||
| 57 | 57 | type Workflow struct { | |
| 58 | Path string | ||
| 58 | Path string | ||
| 59 | // BlobSHA identifies the exact file contents queued for a run, so a workflow | ||
| 60 | // edited after enqueue cannot change what an already-queued job executes. | ||
| 61 | BlobSHA string | ||
| 59 | 62 | Version int | |
| 60 | 63 | Branches []string | |
| 61 | 64 | Jobs []Job | |
| @@ -257,6 +260,36 @@ func firstLine(s string) string { | |||
| 257 | 260 | return s | |
| 258 | 261 | } | |
| 259 | 262 | ||
| 263 | // At loads and validates the workflows in a git tree, with the blob id of each | ||
| 264 | // file so a run can be pinned to the version it was queued from. | ||
| 265 | func At(g *gitx.Repo, commit string) ([]*Workflow, []string) { | ||
| 266 | if commit == "" || commit == gitx.ZeroSHA { | ||
| 267 | return nil, nil | ||
| 268 | } | ||
| 269 | blobs, err := g.LsTreeR(commit) | ||
| 270 | if err != nil { | ||
| 271 | return nil, []string{fmt.Sprintf("cannot read %s: %v", Dir, err)} | ||
| 272 | } | ||
| 273 | files := map[string]string{} | ||
| 274 | shas := map[string]string{} | ||
| 275 | for _, e := range blobs { | ||
| 276 | if !strings.HasPrefix(e.Name, Dir+"/") { | ||
| 277 | continue | ||
| 278 | } | ||
| 279 | data, err := g.BlobBySHA(e.SHA) | ||
| 280 | if err != nil { | ||
| 281 | return nil, []string{fmt.Sprintf("cannot read %s: %v", e.Name, err)} | ||
| 282 | } | ||
| 283 | files[e.Name] = string(data) | ||
| 284 | shas[e.Name] = e.SHA | ||
| 285 | } | ||
| 286 | wfs, problems := Check(files) | ||
| 287 | for _, wf := range wfs { | ||
| 288 | wf.BlobSHA = shas[wf.Path] | ||
| 289 | } | ||
| 290 | return wfs, problems | ||
| 291 | } | ||
| 292 | |||
| 260 | 293 | // ValidateTree checks the workflows in a git tree. Called from pre-receive, so a | |
| 261 | 294 | // bad workflow fails the push that introduced it. | |
| 262 | 295 | func ValidateTree(g *gitx.Repo, commit string) []string { | |