vole

Clone

Read only, no credentials:

https://rodent.software/git/vole.git

Write over SSH with a registered key:

git clone ssh://git@0.0.0.0:2222/vole.git

A free and open-source forge that lives in your Git repo.

Require an image for every CI job

70b702649b875d841e294f5a085ca784d43e5bc4 · Keenan Nemetz · 15d ago · parent 7da9111e

Require an image for every CI job

Runners now require an OCI runtime, and each workflow job must name either a registry image or an image-file definition under .vole/images. Definitions are validated at push time, pinned with the run, built locally by the runner, and cached by definition digest. Records and the run page carry the image and definition digest.

Split the CI guide into workflows and runners, and add reference pages for workflow and image definition fields.

docs/ci.md −302

@@ -1,302 +0,0 @@
	1	# CI
	2
	3	CI should run where the work is, not only where a vendor says it can. Vole sends
	4	jobs over SSH to runners you control: a server, a spare desktop, or a machine in
	5	the next room. The runner executes your steps and reports the result, which is
	6	stored with the repository.
	7
	8	## Add a workflow
	9
	10	Workflows live in `.vole/workflows/*.yaml`. When you push a new workflow, Vole
	11	checks it before accepting the push, so a broken workflow is rejected early.
	12
	13	```yaml
	14	version: 1
	15	on:
	16	push:
	17	branches: [main]
	18	jobs:
	19	build:
	20	runs-on: [linux, arm64] # only a runner with both labels takes this job
	21	steps:
	22	- name: test
	23	run: go test ./...
	24	- name: build
	25	run: make dist
	26	artifacts:
	27	- path: dist/*
	28	pages:
	29	path: site/ # publish this directory as the repository's site
	30	```
	31
	32	Triggers work like this:
	33
	34	- `on.push.branches` limits a workflow to certain branches.
	35	- `on.push.tags` limits it to tag pushes.
	36	- A workflow with neither runs on every branch.
	37	- A workflow that names `tags:` runs only on tags.
	38
	39	When a push starts a job, Vole remembers the exact commit and the exact workflow
	40	file. Editing the workflow later does not change a job that is already queued.
	41
	42	## Contributor branches
	43
	44	A push to `refs/heads/contrib/<name>/...` queues no work by itself. Those
	45	branches contain code from outside the project's maintainer group, and running a
	46	workflow executes it on a runner. CI is held until a maintainer or admin opens the
	47	repository page and chooses Allow CI for this tip next to the contributor
	48	branch.
	49
	50	The approval is keyed by the exact commit, not by the branch. A new commit or a
	51	force-push is a new decision, so the contributor cannot push once and inherit
	52	approval forever. The approval is stored in the CI queue database, so a
	53	reconciliation or a temporary queue outage does not silently run an unapproved
	54	tip.
	55
	56	Allowing CI queues every workflow that applies to the branch. Workflows with no
	57	`on.push.branches` filter run on every branch, including names with slashes such
	58	as contributor branches.
	59
	60	You can check workflows locally before pushing:
	61
	62	```sh
	63	vole workflow check
	64	vole workflow list
	65	```
	66
	67	## Run a workflow locally
	68
	69	You can run the steps without a server or a runner. From the repository root:
	70
	71	```sh
	72	vole workflow run # every job in every workflow
	73	vole workflow run build # every job in build.yaml
	74	vole workflow run ci -j test # one job in one workflow
	75	vole workflow run -j test # the job named test in every workflow
	76	```
	77
	78	A workflow can be named with or without its extension (`ci` or `ci.yaml`). Steps
	79	run in the working tree, in the order the workflow declares them. The command
	80	stops at the first failure and exits with that step's status, so it can wrap a
	81	script. It validates the workflow first, with the same checker a push uses.
	82
	83	A local run is a check of the commands, not a rehearsal of the push. It ignores
	84	`on.push` filters and `runs-on`, and it does not publish artifacts or deploy a
	85	site. It sets `CI=true`, `VOLE_CI=1`, and `VOLE_CI_LOCAL=1`, gives each step its
	86	declared timeout, and reads no standard input. Jobs share your working tree, so
	87	a later job sees files an earlier one wrote; a pushed run starts each job from a
	88	clean checkout.
	89
	90	## Start a runner
	91
	92	Start a runner anywhere it can reach the Vole server:
	93
	94	```sh
	95	vole runner --remote ssh://git@host:2222
	96	vole runner --remote ssh://git@host:2222 --once # run one job, then exit
	97	```
	98
	99	You can describe the machine and limit which repositories it serves:
	100
	101	```sh
	102	vole runner --remote ssh://git@host:2222 --label linux --label arm64
	103	vole runner --remote ssh://git@host:2222 --repos demo --repos docs
	104	```
	105
	106	- `--label` says what the machine is. A job is sent only to a runner that has
	107	every label in its `runs-on`. A job with no `runs-on` can run anywhere.
	108	- `--repos` narrows what the account is already allowed to run. It cannot grant
	109	access to a repository.
	110	- `--dir` chooses where checkouts are kept.
	111
	112	### Run each job in a container
	113
	114	A runner can execute every job's steps in a container built from an image you
	115	choose. The image belongs to the machine, not to the workflow: the workflow still
	116	asks only for labels, and the operator decides what those labels mean.
	117
	118	```sh
	119	vole runner --remote ssh://git@host:2222 \
	120	--image golang:1.27-bookworm@sha256:... \
	121	--container-runtime podman \
	122	--label linux --label go1.27
	123	```
	124
	125	One container is created per job and removed afterward. The workflow's steps run
	126	inside it, but the runner process stays on the host: the SSH key, the trusted Git
	127	cache, and the record push are never exposed to the job. Each job gets a fresh
	128	clone, so a job cannot leave a Git hook or a modified checkout behind for the next
	129	one.
	130
	131	The image is the dependency manifest. It should contain the toolchains and system
	132	packages the builds need, plus `/bin/sh` and a POSIX `sleep` (Vole runs shell steps
	133	and keeps the container alive between them). Pin it by digest; a mutable tag under
	134	the same label silently changes what a build means.
	135
	136	Container defaults are intentionally closed:
	137
	138	- no network (`--container-network none`); add `bridge` only when a build needs it;
	139	- read-only root filesystem, with `/tmp` as a writable tmpfs;
	140	- no capabilities, `no-new-privileges`, and no host mounts except the job tree;
	141	- no Docker socket, SSH agent, or host environment;
	142	- the steps run as the runner user's `uid:gid` by default.
	143
	144	Use `--container-runtime` to choose `docker`, `podman`, or `nerdctl`. The runner
	145	user must be able to talk to that runtime: rootless Podman is preferred, because
	146	membership in the Docker group is effectively root on the host. Do not mount the
	147	host's Docker socket into a job; that would undo the isolation.
	148
	149	`--container-pull` controls whether an image is pulled (`missing`, `always`, or
	150	`never`). `--container-user`, `--container-userns`, `--container-memory`,
	151	`--container-cpus`, and `--container-pids` override the user, namespace mode, and
	152	resource limits. Rootless Podman uses `keep-id` by default so the numeric user can
	153	write the bind-mounted job tree; for rootless Docker, choose a user that matches
	154	the runtime's mapping (container root is the runner user) or use a runtime with
	155	`keep-id`.
	156
	157	Without `--image`, jobs run directly on the runner host, as they did before. That
	158	is useful for a trusted machine, but it is not a sandbox. The run record names the
	159	image and runtime when a container was used, so the run page says what actually
	160	executed.
	161
	162	All runners share one pool. An idle runner waits for work instead of asking
	163	repeatedly, and Vole gives the next job to the repository with the least work in
	164	progress. Set `max_jobs_per_repo` when you need a hard limit per repository.
	165
	166	The `/runners` page shows who is connected, what each runner is building, and how
	167	much work is queued.
	168
	169	## Who can run jobs and report results
	170
	171	A job runs repository code, so Vole uses a runner allowlist for both taking a job
	172	and reporting its result:
	173
	174	```sh
	175	vole admin acl runners demo ci-runner-1 # replace the list
	176	vole admin acl runners demo --add ci-runner-2 # add one account
	177	vole admin acl runners demo --remove ci-runner-1 # remove one account
	178	vole admin acl runners demo --clear # back to admins only
	179	```
	180
	181	Admins always count. For everyone else, a maintainer running CI on their laptop does
	182	not turn a required check green. The result is still recorded, and the run page
	183	says whether it counts.
	184
	185	When a job finishes, the runner uploads its log, writes a record under
	186	`runs/<run id>/<attempt>/`, and pushes the record together with artifact pointers
	187	and any release pointers. Vole checks that the account is allowed, that the record
	188	names that account, and that it describes the job the account was given. The run
	189	page names the account and links to the commit that holds the record.
	190
	191	The runner did the work, so the runner reports the result. The SSH key behind the
	192	push identifies the runner, and `data/audit.log` records the decision.
	193
	194	## Logs
	195
	196	The runner uploads the log before reporting the exit code. Vole does not stream
	197	logs: a log assembled by the server would not be the runner's own record.
	198
	199	```text
	200	/r/<repo>/runs/<id>/log # latest attempt
	201	/r/<repo>/runs/<id>/log?attempt=N # a particular retry
	202	```
	203
	204	If a runner is killed, its attempt has no log.
	205
	206	## Artifacts
	207
	208	Use `artifacts:` to list files that a build should publish. Vole expands the
	209	patterns in the checkout after the steps pass, then uploads the files to the same
	210	storage used by LFS and CI logs.
	211
	212	```yaml
	213	artifacts:
	214	- path: dist/*
	215	```
	216
	217	Artifacts appear on the run page with a download link:
	218
	219	```sh
	220	curl -OJ http://host/r/demo/runs/01M5.../artifacts/tool-linux
	221	sha256sum tool-linux
	222	```
	223
	224	A pattern that matches nothing fails the job. A build that stopped producing the
	225	declared file is exactly what the declaration is meant to catch. If two matches
	226	have the same base name, Vole refuses them so nothing is published under an
	227	unexpected name.
	228
	229	A run artifact belongs to its run. It is not a release artifact. Moving one into a
	230	release is a separate step (`vole release attach`). Run artifacts are kept for as
	231	long as the run exists.
	232
	233	## Build a release in CI
	234
	235	A tag push can build a release. Mark the artifacts `release: true`:
	236
	237	```yaml
	238	version: 1
	239	on:
	240	push:
	241	tags: ['v*']
	242	jobs:
	243	release:
	244	steps:
	245	- name: build
	246	run: make dist
	247	artifacts:
	248	- path: dist/*
	249	release: true
	250	```
	251
	252	```sh
	253	git tag -a v1.0.0 -m "first release" # needs notes and a tagger
	254	git push origin refs/tags/v1.0.0 # the runner builds and publishes
	255	```
	256
	257	The record on the `vole` branch names the build (`release: v1.0.0 artifacts from
	258	run 01M5...`), so `git log vole` shows which run shipped the files. A lightweight
	259	tag is refused: it has no release notes and no tagger. A tag run never counts as a
	260	branch's required check, even if the tag points at that branch's tip.
	261
	262	See [Releases](releases.md) for the release page and artifact rules.
	263
	264	## Publish a website in CI
	265
	266	`pages:` publishes a directory as the repository's website, using the same rules
	267	as `vole pages deploy`: the branch contains exactly those files, so a page removed
	268	from the build stops being served.
	269
	270	```yaml
	271	pages:
	272	path: site/
	273	```
	274
	275	The commit names the build (`site: main from run 01M5... (6beaa53)`), so
	276	`git log pages` shows what put that tree there.
	277
	278	Vole refuses the deploy if the run is no longer the newest commit on its branch.
	279	An older build finishing late must not silently roll the site back. The run log
	280	explains why:
	281
	282	```text
	283	this run built f1f415ec but main is at 6beaa530 now; the site was not deployed
	284	```
	285
	286	Only one job per workflow can publish the site. The branch name is set by the
	287	instance (`pages_ref`), not by the workflow.
	288
	289	## How the queue works
	290
	291	- Finished runs are stored as records on the `vole` branch. `data/runs.db` only
	292	stores work that has not happened yet.
	293	- A push normally reaches the queue in under a second. Every 30 seconds Vole also
	294	compares each branch and tag with the last tip it queued, so a missed
	295	notification still produces a run.
	296	- Pushing again cancels queued runs for the commit it replaced. It never cancels a
	297	run that is already in progress.
	298	- If a runner disappears, its job is requeued. After five attempts the job fails
	299	instead of cycling forever.
	300	- Deleting `data/runs.db` loses attempts that were in flight. Current branch and
	301	tag tips are queued again from Git, and `vole admin index rebuild` restores finished
	302	history from the records on each repository's `vole` branch.