vole
Clone
Read only, no credentials:
https://rodent.software/vole.git
Write over SSH with a registered key:
git clone ssh://git@rodent.software:2222/vole.git
A free and open-source forge that lives in your Git repo.
Phase 4 slice 2: the queue and the runner
b44cbfaf873fda01a458b527e5684b2386266117 · Keenan Nemetz · 18d ago · parent 8fe238d4
Phase 4 slice 2: the queue and the runner A push now runs a pipeline. post-receive spools one file per accepted ref update — the hook must never write to a database or fail a commit — and the server ingests those into data/runs.db, pinning each run to the commit and to the workflow blob it was queued from. The unique index on (repo, ref, sha, workflow, job) is what makes ingestion idempotent, so a spool file read twice after a crash does not run twice. vole runner connects over SSH with the key it would push with: no token, no stored secret, and the same identity the rest of the system uses. It may only run jobs for repositories where its account is an admin, because running a job executes that repository's code — a bigger power than writing to it. Delivery is at-least-once by construction: - A lease hands out a job for a fixed window; the runner heartbeats while it works. - An expired lease means the runner died, and the janitor returns the job to the queue for somebody else. - A runner whose lease is gone stops rather than reporting a result nobody will record, and a job that kills five runners in a row is failed instead of cycling. Three bugs found while building it, each worth the test that now covers it: - Background loops lived in Run(), so the tests — which serve the SSH listener themselves — never had ingestion or lease expiry. They start in New() now. - post-receive derived the repository name from GIT_DIR, which git sets to "." for hooks, so every event was queued for a repository called ".". The same trap had already appeared in a pre-receive message; both now resolve the absolute path. - Leases ordered only by creation time, so two pushes in the same second could be handed out in any order; the id breaks the tie, and ULIDs sort by time. Verified end to end on a live instance: a push queues a job, a runner leases it, clones the commit, prints "hi", and the run closes success with the log attached. The integration tests cover the two exit criteria that matter — a push runs a pipeline, and killing a runner mid-job requeues the work onto a second runner (attempts=2, success) — plus the property that editing a workflow after a push does not change what the already-queued run executes. Not built yet: statuses on change requests and required-check gating (slice 3), and signed run records (Phase 7). Runs live in data/runs.db and are visible there, not on the web.
internal/config/config.go +21−2
| @@ -5,6 +5,7 @@ import ( | |||
| 5 | 5 | "fmt" | |
| 6 | 6 | "os" | |
| 7 | 7 | "path/filepath" | |
| 8 | "time" | ||
| 8 | 9 | ||
| 9 | 10 | "github.com/BurntSushi/toml" | |
| 10 | 11 | ) | |
| @@ -20,6 +21,9 @@ type Config struct { | |||
| 20 | 21 | // ReceiveMaxBytes caps a whole push, enforced by git's own | |
| 21 | 22 | // receive.maxInputSize in each repo rather than by buffering in the proxy. | |
| 22 | 23 | ReceiveMaxBytes int64 `toml:"receive_max_bytes"` | |
| 24 | // LeaseSeconds is how long a runner may hold a job without a heartbeat before | ||
| 25 | // the job is handed to somebody else. | ||
| 26 | LeaseSeconds int `toml:"lease_seconds"` | ||
| 23 | 27 | ||
| 24 | 28 | Path string `toml:"-"` | |
| 25 | 29 | } | |
| @@ -58,8 +62,23 @@ func (c *Config) ReposDir() string { return filepath.Join(c.DataDir, "repos") } | |||
| 58 | 62 | func (c *Config) SystemRepo() string { return filepath.Join(c.DataDir, "system.git") } | |
| 59 | 63 | ||
| 60 | 64 | // AuditLog records privileged operations. Roster history also lives in git. | |
| 61 | func (c *Config) AuditLog() string { return filepath.Join(c.DataDir, "audit.log") } | ||
| 62 | func (c *Config) IndexDB() string { return filepath.Join(c.DataDir, "index.db") } | ||
| 65 | func (c *Config) AuditLog() string { return filepath.Join(c.DataDir, "audit.log") } | ||
| 66 | func (c *Config) IndexDB() string { return filepath.Join(c.DataDir, "index.db") } | ||
| 67 | |||
| 68 | // RunsDB holds the CI queue and finished runs. Unlike index.db it is not derived: | ||
| 69 | // a run happened or it did not. | ||
| 70 | func (c *Config) RunsDB() string { return filepath.Join(c.DataDir, "runs.db") } | ||
| 71 | |||
| 72 | // SpoolDir receives accepted push events from post-receive hooks. | ||
| 73 | func (c *Config) SpoolDir() string { return filepath.Join(c.DataDir, "queue", "spool") } | ||
| 74 | |||
| 75 | // Lease returns how long a runner may hold a job without a heartbeat. | ||
| 76 | func (c *Config) Lease() time.Duration { | ||
| 77 | if c.LeaseSeconds <= 0 { | ||
| 78 | return 60 * time.Second | ||
| 79 | } | ||
| 80 | return time.Duration(c.LeaseSeconds) * time.Second | ||
| 81 | } | ||
| 63 | 82 | func (c *Config) InstanceKey() string { return filepath.Join(c.DataDir, "instance.key") } | |
| 64 | 83 | func (c *Config) SSHHostKey() string { return filepath.Join(c.DataDir, "ssh_host_ed25519_key") } | |
| 65 | 84 | ||