vole

Clone

Read only, no credentials:

https://rodent.software/git/vole.git

Write over SSH with a registered key:

git clone ssh://git@0.0.0.0:2222/vole.git

A free and open-source forge that lives in your Git repo.

Code

392 commit(s) on main

Compare History

Merge change request 01M70WES5ZZDJ6RP226WB11BBZ: Let people rename and remove their own passkeys keenan · 16m ago 4b8c9e1d

source preview history blame raw 10.1 KiB

1	# CI runners
2
3	A runner is a machine that leases jobs from your instance over SSH, runs each job
4	in a container, and reports the result. It is a second machine, never the forge's
5	own: it executes repository code and can read every secret its jobs declare, so it
6	should not share a disk with the account database and the SSH host key.
7
8	Give it its own machine the same way you created the forge — a rendered first-boot
9	file pasted into the console — or set one up on a machine you already have. Either
10	way there is one step only an admin can do, and it is per repository.
11
12	## Create a runner machine
13
14	Create a server the way the [Hetzner](hetzner.md), [DigitalOcean](digitalocean.md),
15	or [AWS](aws.md) guide describes, with three differences: the runner's file instead
16	of the forge's, no inbound ports except 22, and a disk sized for container images.
17
18	```sh
19	cd deploy/cloud-init
20	FORGE_HOST=forge.example.com \
21	VOLE_PACKAGE_URL=https://rodent.software/vole/releases/v0.2.0/artifacts/vole_0.2.0-1_amd64.deb \
22	./render.sh runner > runner-user-data.yaml
23	```
24
25	`render.sh runner` also reads `RUNNER_LABEL_NAMES` (default `linux`),
26	`RUNNER_NETWORK` (default `bridge`), `RUNNER_RUNTIME` (default `podman`),
27	`RUNNER_JOBS` (default `1`, how many jobs run at once), `RUNNER_HOSTNAME`,
28	`RUNNER_MEMORY` (default `4G`, what one job may take from RAM), `RUNNER_PIDS`
29	(default `2048`), `RUNNER_TMPFS` (default `1G`, the job's `/tmp`, which is a tmpfs
30	and therefore memory), and `RUNNER_DISK_RESERVE` (default `2G`, free space kept under
31	`RUNNER_DIR`).
32
33	Size the machine for `RUNNER_JOBS` times `RUNNER_MEMORY`, and keep that below the
34	unit's `MemoryMax=90%` backstop, which bounds the whole runner if a job gets past its
35	own cgroup. `RUNNER_DISK_RESERVE` is what the machine keeps for itself: a job that
36	takes the filesystem under `RUNNER_DIR` below it is stopped while it runs, and the
37	job's log says so. See
38	[Runners § What a job may consume](../runner.md#what-a-job-may-consume) for what each
39	limit means, and [Running several jobs at once](../runner.md#running-several-jobs-at-once)
40	for `RUNNER_JOBS`.
41	The file installs the package, creates an unprivileged
42	`ci-runner` account with the subordinate ids rootless Podman needs, generates the
43	runner's key, pins the instance's host key once, and enables `vole-runner.service`.
44
45	If you would rather do it by hand on a machine that already exists, the steps are
46	below.
47
48	## On a machine you already have
49
50	A runner needs an OCI runtime — rootless Podman is preferred, because membership
51	in the Docker group is effectively root on the host — and it needs to reach the
52	instance's SSH port.
53
54	On a machine you configure yourself, the package carries the unit, so it is three
55	steps: install it, set up the account the runtime needs, and fill in the
56	arguments.
57
58	```sh
59	sudo apt install ./vole_0.2.0-1_amd64.deb # dnf on Fedora and RHEL
60	sudo useradd --create-home --home-dir /var/lib/vole-runner --shell /usr/sbin/nologin ci-runner
61	sudo usermod --add-subuids 100000-165535 --add-subgids 100000-165535 ci-runner
62	sudo apt install podman slirp4netns fuse-overlayfs uidmap
63	sudo install -d -o ci-runner -g ci-runner -m 0700 /var/lib/vole-runner/.ssh
64	sudo -u ci-runner ssh-keygen -q -t ed25519 -N '' -f /var/lib/vole-runner/.ssh/id_ed25519
65
66	# Pin the instance's host key. A runner executes repository code, so a host that
67	# can answer in the instance's place could send it any job.
68	ssh-keyscan -p 2222 git@forge.example.com \| sudo tee /etc/vole/runner_known_hosts
69	```
70
71	Then put the arguments in `/etc/vole/runner.env`. The package ships an example
72	with each option explained:
73
74	```sh
75	sudo tee /etc/vole/runner.env >/dev/null <<'EOF'
76	VOLE_RUNNER_ARGS=--remote ssh://git@forge.example.com:2222 --container-runtime podman --container-network bridge --container-memory 4G --container-tmpfs 1G --disk-reserve 2G --dir /var/lib/vole-runner --known-hosts /etc/vole/runner_known_hosts --label linux
77	EOF
78	sudo systemctl daemon-reload
79	sudo systemctl enable --now vole-runner
80	```
81
82	Add `--jobs N` to that line to run several jobs at once on a machine with room for
83	them, and raise `--container-memory` with it: the memory limit is per job, so a machine
84	running two jobs at `4G` needs room for `8G` below the unit's `MemoryMax` backstop.
85
86	`systemctl stop vole-runner` stops taking work and lets the jobs in flight finish,
87	up to the unit's `TimeoutStopSec` (60 seconds). Raise it in a drop-in if this
88	machine's builds run longer and you would rather wait for them:
89
90	```sh
91	sudo systemctl edit vole-runner # [Service] / TimeoutStopSec=30min
92	```
93
94	`systemctl status vole-runner` and `journalctl -u vole-runner` are where a runner
95	that is not taking work says why.
96
97	## What the package configures in the container runtime
98
99	A rootless runtime needs two settings the distributions do not agree on, so the
100	package installs both as drop-ins. Each merges with the distribution's own file
101	instead of replacing it, and both reach the runner's account: system configuration
102	is read for rootless users too, so nothing has to be written into a home directory.
103
104	\| File \| What it sets, and why \|
105	\|---\|---\|
106	\| `/etc/containers/registries.conf.d/50-vole-runner.conf` \| `unqualified-search-registries = ["docker.io"]`. Debian and Ubuntu ship `registries.conf` entirely commented out, so a base image named as a short name has no registry to resolve against and the build fails with `short-name … did not resolve to an alias and no unqualified-search registries are defined`. The digest in `.vole/images` is the pin; this list only says where to ask. \|
107	\| `/etc/containers/containers.conf.d/50-vole-runner.conf` \| `[engine] cgroup_manager = "cgroupfs"`. `vole-runner.service` is a system service, so the runner's account has no systemd user manager for podman to ask for a cgroup scope. It falls back to sd-bus, which answers `Interactive authentication required`, and the first `RUN` step of an image build fails. `cgroupfs` writes into the cgroup subtree `Delegate=yes` already gives the service. \|
108
109	A site that wants a different registry list writes its own drop-in with a higher
110	number, such as `99-site.conf`, which is read after these.
111
112	The unit also runs with `NoNewPrivileges=false`, alone among the units this package
113	installs. Rootless podman, Docker, and nerdctl all map the calling user into the
114	container through setuid `newuidmap` and `newgidmap`; under `NoNewPrivileges=true`
115	a setuid helper cannot gain privilege, so the mapping fails before a container
116	exists (`newuidmap: write to uid_map failed: Operation not permitted`). The
117	isolation a job gets is its container, not the unit's sandbox, so the rest of the
118	hardening stays.
119
120	If you would rather give podman the systemd user manager it prefers, run the
121	runner as a user unit instead: `loginctl enable-linger ci-runner`, a unit in
122	`ci-runner`'s own `systemctl --user` session, and no `cgroup_manager` setting. That
123	session lives at `/run/user/<uid>`, so the unit's `RuntimeDirectory=vole-runner` and
124	`XDG_RUNTIME_DIR=/run/vole-runner` have to go with it — the two arrangements cannot
125	both be assumed, and this one is not what the shipped unit does.
126
127	## Register the runner, then allow it on a repository
128
129	The runner's key is not an admin key. It becomes a bot account, and the
130	allowlist decides what it may build:
131
132	```sh
133	# The public key is on the runner; this copies it to the instance as a new bot.
134	ssh root@<runner-address> cat /var/lib/vole-runner/.ssh/id_ed25519.pub \
135	\| vole admin --remote ssh://git@forge.example.com:2222 \
136	account create ci-runner-1 --bot --key -
137
138	# Per repository: let it take jobs, and give it read so it can clone.
139	vole admin --remote ssh://git@forge.example.com:2222 acl runners add demo ci-runner-1
140	vole admin --remote ssh://git@forge.example.com:2222 acl set-role demo ci-runner-1 read
141	```
142
143	This is the part that cannot happen at first boot: repositories do not exist yet,
144	and the allowlist is per repository. Add each new repository to it as you create
145	it, or the runner will sit idle while its jobs wait in the queue.
146
147	A bot cannot sign in to the browser and cannot be an instance admin. That second
148	part matters: admins count as trusted reporters, so a bot that was an admin would
149	make every result count whatever the allowlist said.
150
151	## Check that it works
152
153	- The Admin page's Runners tab (`/-/admin?tab=runners`) shows what is connected,
154	what each runner is building, and how much is queued.
155	- Push a commit on a branch with a workflow. The run page shows which runner took
156	it and whether the result counts.
157	- A run that executed on a runner outside the allowlist still exists and still
158	says so; it just does not count for the branch.
159
160	## What to know before you give it a repository
161
162	A runner can read every secret its jobs declare. That is what running the job
163	means: the values arrive on the machine. If the machine is not yours, the secrets
164	are not either. This is also why an instance admin's machine should not be a
165	runner for a repository whose secrets it should not see.
166
167	The allowlist is the control, not the container. A job that escapes its
168	container finds a machine that can reach the instance and the internet. Keep the
169	runner on its own machine, run it as an unprivileged account, and give it no
170	credentials except the one key.
171
172	Network is a per-job decision this machine permits. A `bridge` runner still
173	gives a job that asks for none no network. What it cannot do is take the choice
174	away from a job, so a workflow you approve for a contributor branch is a workflow
175	that may reach the internet with whatever it can read.
176
177	Disk fills where builds happen. Container images, the `/cache` mount, and the
178	saved checkouts live under `--dir`. `--cache-size` (5 GiB) and `--git-cache-size`
179	(10 GiB) bound the last two; the images are the runtime's own storage, so size the
180	machine for them.
181
182	## Upgrading
183
184	The runner is the same package as the instance, and the two talk over a versioned
185	protocol. Upgrade the instance first, then the runner machines:
186
187	```sh
188	sudo apt install ./vole-<new-version>.deb
189	sudo systemctl restart vole-runner
190	```
191
192	## Next
193
194	- [Runners reference](../runner.md) — every option, the container defaults, caching,
195	and how a lease is reported
196	- [Workflows](../workflows.md) — what a job asks a runner for, including `runs-on`
197