vole
Clone
Read only, no credentials:
https://rodent.software/git/vole.git
Write over SSH with a registered key:
git clone ssh://git@0.0.0.0:2222/vole.git
A free and open-source forge that lives in your Git repo.
Phase 4 slice 2: the queue and the runner
b44cbfaf873fda01a458b527e5684b2386266117 · Keenan Nemetz · 17d ago · parent 8fe238d4
Phase 4 slice 2: the queue and the runner A push now runs a pipeline. post-receive spools one file per accepted ref update — the hook must never write to a database or fail a commit — and the server ingests those into data/runs.db, pinning each run to the commit and to the workflow blob it was queued from. The unique index on (repo, ref, sha, workflow, job) is what makes ingestion idempotent, so a spool file read twice after a crash does not run twice. vole runner connects over SSH with the key it would push with: no token, no stored secret, and the same identity the rest of the system uses. It may only run jobs for repositories where its account is an admin, because running a job executes that repository's code — a bigger power than writing to it. Delivery is at-least-once by construction: - A lease hands out a job for a fixed window; the runner heartbeats while it works. - An expired lease means the runner died, and the janitor returns the job to the queue for somebody else. - A runner whose lease is gone stops rather than reporting a result nobody will record, and a job that kills five runners in a row is failed instead of cycling. Three bugs found while building it, each worth the test that now covers it: - Background loops lived in Run(), so the tests — which serve the SSH listener themselves — never had ingestion or lease expiry. They start in New() now. - post-receive derived the repository name from GIT_DIR, which git sets to "." for hooks, so every event was queued for a repository called ".". The same trap had already appeared in a pre-receive message; both now resolve the absolute path. - Leases ordered only by creation time, so two pushes in the same second could be handed out in any order; the id breaks the tie, and ULIDs sort by time. Verified end to end on a live instance: a push queues a job, a runner leases it, clones the commit, prints "hi", and the run closes success with the log attached. The integration tests cover the two exit criteria that matter — a push runs a pipeline, and killing a runner mid-job requeues the work onto a second runner (attempts=2, success) — plus the property that editing a workflow after a push does not change what the already-queued run executes. Not built yet: statuses on change requests and required-check gating (slice 3), and signed run records (Phase 7). Runs live in data/runs.db and are visible there, not on the web.
docs/roadmap.md +15−5
| @@ -245,9 +245,18 @@ names, timeouts beyond the cap, files in subdirectories, non-YAML names. `vole w | |||
| 245 | 245 | check` runs the same validator locally. Workflows carry `version`, branch filters | |
| 246 | 246 | (globs, default all) and jobs of shell steps with per-step timeouts. | |
| 247 | 247 | ||
| 248 | ✅ **Slice 2 — queue and runner.** `post-receive` spools one file per accepted ref | ||
| 249 | update; the server ingests them into `data/runs.db` (unique on repo+ref+sha+workflow+job, | ||
| 250 | so re-reading the spool never double-queues) and pins each run to the commit and the | ||
| 251 | workflow blob it came from. `vole runner` connects over SSH with the user's key — no | ||
| 252 | token — long-polls for a lease, checks the commit out, runs the steps with per-step | ||
| 253 | timeouts, heartbeats while it works and reports the exit code and log. A lease that | ||
| 254 | expires returns the job to the queue; a job that kills five runners is failed. | ||
| 255 | |||
| 248 | 256 | Remaining: | |
| 249 | 257 | ||
| 250 | - Queue and runner (next slice, below) | ||
| 258 | - Statuses on change requests and required-check gating (slice 3) | ||
| 259 | - Run records committed to a ref and signed by the instance key (Phase 7) | ||
| 251 | 260 | - Push-event queue persisted in SQLite, with at-least-once delivery | |
| 252 | 261 | - Runner protocol: `vole runner` registers, long-polls for work, leases a job, heartbeats; | |
| 253 | 262 | an expired lease requeues the job | |
| @@ -257,10 +266,11 @@ Remaining: | |||
| 257 | 266 | - Statuses surfaced on CRs and satisfying required-check gating from Phase 3 | |
| 258 | 267 | - Log and artifact storage onto the content-addressed store (shared with Phase 5) | |
| 259 | 268 | ||
| 260 | **Exit criteria:** a push runs a pipeline end-to-end; killing a runner mid-job results in | ||
| 261 | the job being requeued and completing on another runner; a run record can be verified | ||
| 262 | against the SHA it claims; a workflow that mutates its own definition mid-run still executes | ||
| 263 | the version it was queued with. | ||
| 269 | **Exit criteria:** a push runs a pipeline end-to-end ✅; killing a runner mid-job results in | ||
| 270 | the job being requeued and completing on another runner ✅; a workflow that mutates its | ||
| 271 | own definition mid-run still executes the version it was queued with ✅ (steps are pinned | ||
| 272 | at enqueue); a run record can be verified against the SHA it claims — the record carries | ||
| 273 | it, but signing is Phase 7. | ||
| 264 | 274 | ||
| 265 | 275 | ### Phase 5 — LFS | |
| 266 | 276 | ||