vole

Clone

Read only, no credentials:

https://rodent.software/vole.git

Write over SSH with a registered key:

git clone ssh://git@rodent.software:2222/vole.git

A free and open-source forge that lives in your Git repo.

Code

395 commit(s) on main

Compare History

update readme Keenan Nemetz · 25m ago 32290114

source preview history blame raw 16.6 KiB

1	# Operations
2
3	A forge you own should be easy to back up and recover. Vole keeps the important
4	state small: an account database, a handful of files, and ordinary Git repositories.
5	These are the tasks that keep an instance healthy.
6
7	## Audit log
8
9	`data/audit.log` records every privileged action — roster and key changes, access
10	rules, forced lock releases, forced pushes, release rewrites, admin commands,
11	sessions, and ordinary pushes — as one tab-separated line:
12
13	```text
14	timestamp action actor subject detail
15	```
16
17	- One shape. Every writer goes through the same code, so a parser can split
18	on tabs. `action` is a lower-case dotted name (`acl.update`, `push.force`,
19	`session.create`, `admin`); `actor` is the account or OS user, `-` when there
20	is none; `subject` is the repository, ref, or account the action was about.
21	- Fields are escaped and truncated. A key comment, an LFS lock path, a
22	repository name, or an admin argument cannot add a column or a line, and a
23	field is capped at 4 KiB.
24	- Mode 0600, owner only. The file is created that way and tightened on every
25	write, so an instance upgraded from an older version stops being world-readable
26	the first time it writes. Startup refuses a log that is a symlink, a directory,
27	a non-regular file, or owned by another account.
28	- Privileged changes fail closed. A roster, key, access-rule, session, forced
29	lock release, or admin command that cannot be recorded is refused, and the
30	command says so: `the audit log could not be written, so nothing was changed`.
31	A push that overrides a lock, an approval, a release, or history is refused
32	unless its override can be written. There is no setting that turns this off,
33	because an override is exactly what an attacker with config access would set.
34	- Ordinary pushes and web writes are reported, not refused. Those changes
35	are already durable in Git; a failed write appears as `audit log write failed`
36	in the server log (or on the pusher's terminal for `post-receive`) rather than
37	making a working push look rejected.
38	- A broken log stops the instance. `vole serve` and `vole admin` refuse to
39	start when the log cannot be opened for append, which turns a silent, long-lived
40	gap into a startup failure with a path in it.
41
42	An instance admin can read the log in the browser at Admin → Audit log. The
43	page shows one record per row, newest first, and pages back through the file. The
44	Admin overview repeats the newest few, so a review can start there. Both are
45	read-only views: they never edit, truncate, or rewrite a line, and a record the
46	reader cannot parse is skipped rather than hidden from the rest of the page.
47
48	Ship the file off the host — a read-only mount, a log shipper, or a periodic
49	`rsync` of an append-only copy — and alert on the absence of expected lines
50	as well as on the presence of unexpected ones. Local append-only storage is not
51	tamper-evident: anyone who can write `data/` can edit it, so keep an off-host copy
52	that the service account cannot reach.
53
54	## Back up an instance
55
56	Back up the instance state:
57
58	```sh
59	vole admin account backup backup/users.db # accounts, keys, sessions, rules, locks, views
60	sqlite3 data/runs.db ".backup 'backup/runs.db'" # CI history, queue, and approvals
61	cp -a data/logs backup/ # CI attempt logs
62	cp data/ssh_host_ed25519_key backup/ # the SSH host key
63	sqlite3 data/secrets.db ".backup 'backup/secrets.db'" # sealed repository secrets
64	cp data/secrets.key backup/ # the key those secrets need
65	cp -a data/acme backup/ 2>/dev/null # TLS account key and certificates, if ACME is on
66	cp data/audit.log backup/ 2>/dev/null # history of admin actions
67	```
68
69	`vole admin account backup` takes a consistent copy, even while Vole is running. Copying
70	`users.db` by hand can miss recent changes that have not been written to the main
71	database file yet. Do the same for `runs.db` with SQLite's backup command; copying it
72	by hand while the server is writing can miss WAL pages.
73
74	Keep `data/secrets.key` apart from the copies of `data/`. A backup that holds both
75	the database and its key is a backup of plaintext: the encryption protects a copy
76	that loses the key, not one that keeps it. Restoring `data/secrets.db` beside a
77	newly generated key reads nothing, and Vole says so rather than handing a job an
78	empty value.
79
80	If you lose the account database, you need to recreate accounts and re-enroll
81	their passkeys. You do not lose repositories. The backup contains account metadata,
82	public keys, and hashes of session and enrollment tokens; it does not contain a
83	usable password or a plaintext session token. Treat it as private anyway.
84
85	Repositories are ordinary Git repositories under `data/repos/`. You can back them
86	up by copying or cloning. Issues, change requests, and releases travel with them
87	because they are stored in Git. CI history does not: it lives in `data/runs.db` and
88	must be backed up with it.
89
90	## Restore an instance
91
92	1. Restore `data/users.db`, `data/runs.db`, `data/logs/`,
93	`data/ssh_host_ed25519_key`, `data/audit.log`, `data/secrets.db` with the
94	`data/secrets.key` that sealed it, and `data/acme/` if the instance obtains its
95	own TLS certificates.
96	2. Restore the repositories under `data/repos/` and the object store under
97	`data/lfs/` if you keep CI artifacts.
98	3. Run `vole admin index rebuild`.
99	4. Run `vole admin account revoke-sessions --all` so sessions from the restored copy
100	cannot outlive their cookies.
101	5. Start `vole serve`.
102
103	Release notes for an upgrade: an older `users.db` is upgraded in place when it is
104	opened. Accounts used to have an internal id and an optional display label; both
105	are gone, and every key, passkey, enrollment link, session and admin grant is rebuilt
106	around the registered name. Access-rule entries and file locks in that database
107	are rewritten through the same mapping, and the retired `user_names` alias table
108	is dropped. Take a backup first if you might roll the binary back. Approval files
109	live in Git and are not rewritten; an approval written under an old account id
110	stops matching, so ask for a fresh approval.
111
112	Contributor branches moved from `refs/heads/contrib/<account-id>/` to
113	`refs/heads/contrib/<name>/`. Old refs stay in the repository and can still be
114	fetched, but only a maintainer can push them; existing change requests still name
115	the same head commit. A contributor who wants to keep working pushes the same
116	commit under their new folder and opens a request from it.
117
118	Large files and build artifacts live in `data/lfs/`; CI logs live in `data/logs/`.
119	Back both directories up too if you need those files. `vole admin gc` treats files
120	reachable from a repository or a published artifact as things to keep; logs are
121	ordinary files and are not swept.
122
123	## Restart the server
124
125	A clean stop (`systemctl restart`, or Ctrl-C on a development instance) stops
126	handing out new CI jobs and waits up to a minute for the jobs already leased to
127	report their result. A job that finishes in that window is stored and is not run
128	again. A job still running when the window ends loses its runner connection and
129	is requeued as another attempt once its lease expires. The wait is best effort;
130	start long builds well before a planned restart.
131
132	## Clean up stored files
133
134	```sh
135	vole admin gc --dry-run # show what would be removed
136	vole admin gc # remove unused files
137	vole admin gc --grace 24h # keep unused files for a day
138	```
139
140	`vole admin gc` cleans the shared storage used by LFS and build artifacts. CI logs are
141	ordinary files under `data/logs/` and are not part of the sweep. It
142	also repairs each repository's links into that storage, so it helps when files
143	are missing or links are broken.
144
145	The default grace period is one hour. A recent upload that has not been pushed yet
146	will not be removed.
147
148	You can run `vole admin gc` while people are pushing. There is no need to schedule it;
149	storage only grows when someone pushes files.
150
151	## Rebuild indexes
152
153	`data/index.db` is generated by Vole and can be deleted safely:
154
155	```sh
156	vole admin index rebuild
157	```
158
159	`data/runs.db` holds the CI queue, finished-run records, and contributor-branch CI
160	approvals. It is a source of truth, not a rebuildable index: deleting it loses CI
161	history as well as attempts that were in progress. Back it up with the rest of the
162	instance. `vole admin index rebuild` rebuilds only the repository and issue index;
163	it cannot restore CI history.
164
165	## Security notes
166
167	- Two long-term secrets are stored: the SSH host key, and the key that seals
168	repository secrets. Keep both readable only by the Vole user (`0600`). Browser
169	session and enrollment tokens are stored as hashes.
170	- Repository secrets are encrypted at rest, which protects a copy of `data/` that
171	leaves the host. It does not protect a host somebody already owns: file modes
172	are the boundary, and the default key location is beside the database. Anything
173	that can read the data directory can read the values.
174	- Vole creates its own secrets at `0600` and its data directory at `0700`: the
175	roster (`users.db`), the audit log, the SSH host key and the ACME account
176	directory. It also tightens an existing instance on startup, and logs the paths
177	whose mode it changed, so an upgrade from an earlier version cannot leave a
178	private key world-readable. If a mode cannot be changed, startup fails rather
179	than serving keys it cannot protect.
180	A site directory that is deliberately shared with a backup group is fine at
181	`0750`: group access is left alone, world access is not.
182	- Nothing Vole writes is signed: not pushes, not approvals, not CI results. Vole
183	relies on the SSH key at the connection, the checks before a push is accepted,
184	and the audit log.
185	- The web interface is read-only for anonymous visitors. Passkey sign-in enables
186	issue, comment, and approval writes; HTTP Git clone remains read-only. There is
187	no password store. Restoring an old `users.db` should be followed by
188	`vole admin account revoke-sessions --all`. The landing page introduction is
189	not an exception: it is `profile/README.md` in the instance repository
190	(`instance_repo`), edited by pushing like every other customization.
191	- Account names are permanent. They name contributor branches, so an upgrade from
192	a build where names could change keeps each account's current name and forgets
193	the former ones.
194	- Open registration creates accounts without email verification. Leave
195	`open_registration` off unless the instance is meant to accept public
196	contributions.
197	- Remote administration uses the same SSH listener and key. Only active instance
198	admins can run it, and every admin command is audited.
199	- Private repositories return `404` on the web and over HTTP, so visitors cannot
200	confirm that they exist.
201	- Page-view metrics are daily totals keyed by account, plus one anonymous total.
202	They store no path, address, referrer, or user agent; the admin-only status
203	page uses them only to show how much the instance is read.
204	- Vole can obtain and renew TLS certificates itself when `https_addr` is set;
205	otherwise put it behind a proxy that terminates TLS and keeps the `Host`
206	header, so site routing still works. See
207	[Configuration § Automatic HTTPS with ACME](configuration.md#automatic-https-with-acme).
208
209	## Production baseline
210
211	A public instance is a web server, a Git server and a policy engine in one
212	process. The settings below are the ones that matter on the day it is exposed;
213	the linked sections explain each in full.
214
215	- Web behind TLS, on loopback. Leave `https_addr` empty, bind
216	`http_addr = "127.0.0.1:8080"`, and put a proxy in front that terminates TLS
217	and passes the `Host` header through unchanged. Alternatively set `https_addr`
218	with `acme_email` and `acme_domains` and let Vole do it. Set `public_url` to
219	the canonical `https://` address so links and passkeys agree with it. When the
220	request arrives over TLS, or `public_url` is `https://`, Vole sends HSTS on
221	every response, not only on the pages that use passkeys. See
222	[Configuration](configuration.md).
223	- Compression is already here. Forge pages, hosted sites, the stylesheet,
224	scripts and source blobs are gzipped when the client asks for it, so a proxy in
225	front should not compress them again: double compression costs CPU and hides
226	the `ETag` revalidation that keeps the shared scripts and each Prism grammar to
227	one download per browser. Archives, LFS objects, Git responses and ranged
228	downloads are passed through untouched.
229	- Shared caches. If a caching proxy or CDN sits in front of the instance, it
230	may keep public pages, static assets, and artifacts from public repositories.
231	Private pages and downloads are marked `private` or `no-store`, so a cache that
232	honours those directives will not serve them to an anonymous reader. Do not
233	strip or rewrite those headers at the proxy.
234	- SSH firewalled. The SSH listener is the only write transport and the only
235	place an account is authenticated. Expose it to the people who push, not to the
236	internet at large, and leave it on a port other than 22 unless you also run
237	`sshd` elsewhere.
238	- No development shortcuts. `dev_only` and `dev_login` are refused together
239	with a public address, and an instance that needs neither should set neither.
240	The same goes for `open_registration`, which is off by default.
241	- Quotas. Every repository is bounded, so one contributor or maintainer cannot
242	make every page slow or fill the disk: `repo_max_refs` (500), `repo_max_metadata_entries`
243	(2000) and `repo_max_metadata_bytes` (32 MiB) of documents on the metadata branch
244	(issues, requests, comments, approvals, and release records),
245	`repo_max_bytes` (5 GiB) of Git objects, and `repo_max_lfs_bytes` (10 GiB) of
246	LFS objects and artifacts. The checks run in pre-receive, so a refused push
247	moves no refs, and the object bound is enforced on upload. When one bites, the
248	message names it: raise it in `vole.toml`, delete the branches or metadata that
249	are not wanted, or run `vole admin gc` to drop unreachable objects. Per-account
250	browser writes are budgeted for the untrusted (`contribute`) role too
251	(`contributor_writes_per_hour`, 300); a person discussing never reaches it, and
252	maintainers and admins are trusted roles that are not budgeted at all.
253	- Conservative limits. The defaults are chosen for a small instance:
254	`git_http_max_concurrent = 16`, `git_http_max_per_client = 8`,
255	`ssh_max_connections = 64`, `ssh_max_git_processes = 16`,
256	`ssh_handshake_timeout_seconds = 20`, `max_blob_bytes = 100 MiB`,
257	`receive_max_bytes = 1 GiB`, and a 600 s budget for one clone. Raise them with
258	measurements, not with optimism; the read path is anonymous, so a limit that is
259	too low is a slow clone and a limit that is missing is an outage.
260	- Bounded sign-in. The passkey ceremonies are the only anonymous endpoints
261	that cost the instance work, so they are rate-limited per client address
262	(`20` begins a minute), per account name (`10`), and for registrations (`5`),
263	and the in-flight ceremony map is capped at `4096` entries with a five-minute
264	expiry. A refused client gets `429` with `Retry-After`. Behind a proxy, set
265	`git_http_client_ip_header` so the budgets follow the real address. Invite
266	tokens live in the URL fragment, so they never reach an access log; links in
267	the old `/-/invite/<token>` shape are redirected into it. See
268	[Accounts § Browser sign-in](accounts-and-access.md#browser-sign-in).
269	- Bounded before authentication. The SSH listener refuses connections past
270	`ssh_max_connections` before the handshake and drops a connection that has not
271	finished its handshake in `ssh_handshake_timeout_seconds`, so an anonymous
272	client cannot hold a file descriptor by opening a socket and saying nothing.
273	After the handshake one connection may hold eight session channels, and the
274	git commands they run share `ssh_max_git_processes`; an exec line longer than
275	4 KiB and a runner request line longer than 1 MiB are refused. Address-based
276	backoff for repeated bad keys is a quota concern, not a transport bound, so it
277	belongs with the abuse controls rather than here.
278	- Audited administration. Remote administration uses the same SSH listener
279	and is restricted to instance admins. Ship the audit log off the host: local
280	append-only is not tamper-evident. See [Audit log](#audit-log) and
281	[Architecture § Trust model](architecture.md).
282	- Backups you have restored once. `users.db` is not derived and not
283	rebuildable; everything else either is, or can be re-pushed. Test the restore,
284	not the backup. See [Back up an instance](#back-up-an-instance).
285	- The service unit. The packaged unit drops all capabilities, sets
286	`UMask=0027`, and hides other processes and devices. If you serve HTTPS or SSH
287	on a port below 1024, grant `CAP_NET_BIND_SERVICE` with a drop-in rather than
288	running the service as root. See [Packaging](packaging.md).
289