Sign in

Search issue titles and bodies, change requests and comments. Private repositories you cannot read are not included.
vole
Clone

Read only, no credentials:

https://rodent.software/vole.git

Write over SSH with a registered key:

git clone ssh://git@rodent.software:2222/vole.git

A free and open-source forge that lives in your Git repo.

#201

Add deployment files and guides for self-hosted instances

merged · by keenan 3d ago · merged into main at 9bce3723 by keenan 3d ago

keenan opened this 3d ago

Work for #200. Deploying Vole to a server is now one command per cloud, one rendered file for everything else, and one container for host-based setups.

What a user does

On Hetzner Cloud, DigitalOcean, or AWS:

cd deploy/terraform/hetzner
terraform init
terraform apply -var domain=forge.example.com \
  -var "admin_ssh_public_key=$(cat ~/.ssh/id_ed25519.pub)" \
  -var vole_package_url=<the .deb for this architecture>

Three variables, no file to edit, and about three minutes later https://forge.example.com is up with one instance admin whose key is the one you passed. Firewall, elastic/floating address, and the DNS record (when the zone is in the same account) come with it.

Everything else — Vultr, Linode, Scaleway, a spare machine — renders the same first-boot file with deploy/cloud-init/render.sh and pastes it into the provider’s user-data field. A container host uses deploy/docker/, where the entrypoint runs vole admin init once because an image cannot install at run time.

Why the pieces are shaped this way

One setup script, three clouds. deploy/cloud-init/vole.yaml is the only place the steps live; Terraform renders it with templatefile() and render.sh with envsubst. Neither has its own copy, so a fix lands everywhere.

The package installs before the configuration is written. The .deb’s postinstall runs vole admin init against the shipped config, which creates the database and the repository tree at /var/lib/vole/data. Writing the operator’s config first would put a file under a path dpkg owns and risk it being moved aside; writing it second is the documented way to configure an instance, and init is idempotent so a second run is a no-op.

Two TLS arrangements, chosen by one variable. acme (default) lets Vole hold its own certificate on 80/443 and installs the one-capability drop-in the hardened unit needs; proxy serves plain HTTP on loopback for operators who already run a TLS proxy. Certificate issuance is lazy, so a machine booted before DNS resolves does not enter a restart loop — the first visit after DNS answers does the issuance.

No machine images. A golden image per cloud needs a publishing pipeline and a per-cloud import, and goes stale between releases. The .deb install takes about thirty seconds at first boot and installs exactly the version named. That is why deploy/ has no Packer file.

apply does not upgrade. users.db migrates in place the first time a newer binary opens it and cannot be rolled back, so the guides keep upgrades as a package install with a backup first, and Terraform ignores user_data after creation rather than replacing a machine that holds a team’s repositories.

Checks

scripts/check-deploy.py runs in the verify workflow:

  • every YAML under deploy/ parses;
  • the template renders using the placeholder list render.sh uses, with nothing left behind and every promised file present;
  • the boot script and the container entrypoint are valid sh and executable;
  • the compose file’s build context, Dockerfile, and every COPY source exist.

Terraform is formatted with terraform fmt -check, from 1.17.1 pinned by checksum for both amd64 and arm64 in the verify image, the way nfpm and govulncheck are pinned in the steps.

Rehearsed, not assumed:

  • built the .deb and ran the full first-boot script in debian:13-slim — package install, config install, account created as instance admin, and a second run that changes nothing;
  • started the server with both staged configurations: the proxy config answers 200 ok on /healthz, and the ACME config listens on 2222/80/443 with automatic HTTPS enabled;
  • built the container image, created an account in it, and served /healthz from a named volume owned by the image’s user;
  • terraform validate passes in all three configurations;
  • vole workflow run verify passes, including the new step.

Open question, not blocking

The guides take the .deb URL from “a release page” and the examples use example.com. The project’s release artifacts are attached to a release on its own instance, and no stable public download URL is documented, so the guides describe the shape (.../releases/<tag>/artifacts/vole_<version>-1_<arch>.deb) and leave the host to the operator. Worth deciding before this is advertised as a one-command deployment.

Discussion (5)

keenan 3d ago

Added a runner to this change, in two commits after the initial review request.

-var runner=true now creates a second machine beside the forge, and the package carries a vole-runner.service so the runner is something you enable rather than something you babysit in a terminal.

The parts worth a reviewer’s attention:

The unit is inert until configured. It reads /etc/vole/runner.env and an ExecStartPre refuses to start with runner is not configured: set VOLE_RUNNER_ARGS in /etc/vole/runner.env. The alternative — a usage-error restart loop every 10 seconds — makes journalctl useless exactly when someone is looking at it. The arguments live in that file, not in the unit, so one unit serves every machine; systemd splits an unquoted $VOLE_RUNNER_ARGS at spaces, which is the expansion the systemd docs describe for this.

A dedicated account, not the packaged vole account. Rootless Podman needs a home directory and a range of subordinate ids, and the packaged account has neither on purpose. It also means a job that escapes its container does not find the forge’s account on the same disk. The account’s setup is useradd + usermod --add-subuids/--add-subgids, guarded so a second run does not duplicate the range.

The host key is pinned once, never re-scanned. ssh-keyscan at first boot with a retry loop, into /etc/vole/runner_known_hosts, and --known-hosts points there. Re-scanning at every start would trust whatever answers, which is the boundary docs/runner.md says a runner must not cross. If the instance was not reachable within the retry window, the boot log says to run systemctl restart vole-runner once it is, rather than silently trusting a later answer.

Registration stays manual, and it is the only part that could be automated. The allowlist is per repository and none exist at first boot, so the boot script cannot finish the job. terraform output runner_setup prints the three commands with this apply’s addresses:

ssh root@<runner> cat /var/lib/vole-runner/.ssh/id_ed25519.pub \
  | vole admin --remote ssh://git@<domain>:2222 account create ci-runner-1 --bot --key -
vole admin --remote ssh://git@<domain>:2222 acl runners add <repo> ci-runner-1
vole admin --remote ssh://git@<domain>:2222 acl set-role <repo> ci-runner-1 read

Three files describe one runner, so CI checks that they agree. check-deploy.py now renders both templates and asserts that the template’s arguments, the unit’s ExecStart/EnvironmentFile, and the packaged example all name the same things VOLE_RUNNER_ARGS and the same known_hosts path. Drift there produces a machine that is enabled, running, and taking nothing — which looks exactly like a runner that is working.

Rehearsed: systemd-analyze verify on the unit (the only complaint is that /usr/bin/vole is not installed on this host), the account setup and key generation twice in debian:13-slim with no duplicate subuid range, the ExecStartPre message with and without arguments, the .deb contents, terraform validate in all three configurations, and vole workflow run verify in full.

Defaults I chose that are reviewer decisions: runner_network defaults to bridge (most real jobs fetch dependencies, and a bridge runner still gives a job that asks for none no network), and the runner defaults to the same machine size as the forge, with runner_type to raise it for builds.

keenan 3d ago

Dropped the Terraform configurations as asked. deploy/ is now cloud-init and containers only.

What changed:

  • The three Terraform trees are gone, with their provider lock files.
  • The verify image loses the pinned Terraform download (and unzip and gettext-base, which existed for it), and the deployment files step is now just python3 scripts/check-deploy.py — no network, no credentials, no tool to pin.
  • .gitignore and the link checker lose their .terraform entries.
  • deploy/README.md states why: a first-boot file is the same work Terraform described, without a provider, a state file, or a cloud token with account write access.

The guides are rewritten rather than patched, because the shape changed:

  • Index (docs/deploy/README.md) leads with “a rendered file, pasted into a console”, and gains a section that did not exist before: first boot happens once. Editing the file afterwards does nothing, so a later change is $EDITOR /etc/vole/vole.toml + systemctl restart vole, and a first-boot change means a new machine.
  • Hetzner, DigitalOcean, AWS are console walkthroughs now: which image, which size, where the user-data field is, and a firewall table — Hetzner and DigitalOcean leave a new machine open, AWS gives it a group that allows nothing inbound, so that table is load-bearing rather than advice.
  • CI runners leads with render.sh runner and keeps the by-hand path under it. The terraform output runner_setup shortcut is gone, so the registration commands are written out.
  • Any server no longer frames itself as the fallback for providers “without a Terraform guide”.

The .deb already carries vole-runner.service and /etc/vole/runner.env, so the runner path was unaffected.

vole workflow run verify passes on the rebuilt image, check-links and check-deploy pass locally, and render.sh still produces working user data for both templates.

keenan 3d ago

Correction: the deletion commit I pushed an hour ago did not contain the deletions, and the UI was right.

What happened: after git rm -r deploy/terraform I ran git reset to unstage a batch of edits and then re-added the paths I wanted, and deploy/terraform was not in that list. The trees stayed in the branch tip, so the change request correctly showed them as added against main. The working tree had them deleted, which is why my own find and check-deploy run looked clean — they read the working tree, not the pushed commit. The docs rewrite was left uncommitted for the same reason.

Fixed by amending the deletion commit and adding the docs as its own commit, then force-pushing the branch (--force-with-lease). Head is now 942c93c, merge clean, no approvals were affected.

Net diff against main is now:

.dockerignore                                   A
.gitignore                                      M
.vole/images/verify.yaml                        M
.vole/workflows/verify.yaml                     M
README.md                                       M
deploy/README.md                                A
deploy/cloud-init/render.sh                     A
deploy/cloud-init/runner.yaml                   A
deploy/cloud-init/vole.yaml                     A
deploy/docker/*                                 A
docs/README.md                                  M
docs/deploy/*                                   A
docs/packaging.md                               M
docs/runner.md                                  M
packaging/nfpm.yaml                             M
packaging/runner.env                            A
packaging/systemd/vole-runner.service           A
scripts/check-deploy.py                         A

No Terraform files, in either direction.

The check I should have run before pushing, and have run now: git diff --name-only origin/main HEAD on the committed tree rather than trusting a check that reads the working tree.

keenan 3d ago

The download placeholder is resolved: the guides now name https://rodent.software/r/vole/releases as where Vole’s own packages come from, which closes the open question in the description. Head is c1edfd3.

Changed, in docs/deploy/ and the checker’s sample:

VOLE_PACKAGE_URL=https://rodent.software/r/vole/releases/v0.2.0/artifacts/vole_0.2.0-1_amd64.deb
curl -fsSLO https://rodent.software/r/vole/releases/v0.2.0/artifacts/vole_0.2.0-1_amd64.deb

and each cloud guide’s “What you need” now links the release page, so a reader who has never seen this project can find the file instead of being told one exists.

Deliberately not changed: forge.example.com, you@example.com, and the rest of the example.com names. Those stand for the reader’s own instance, not for a public host, and a guide that told someone to run HOSTNAME=rodent.software would be inviting them to deploy a forge on someone else’s domain.

Also left alone, because they are not placeholders: homepage: and Documentation= in packaging/, and the README’s GitHub link. They still point at github.com/rodent-software/vole. If the forge is the project’s public face now, that is a separate decision and a one-line change in each; say the word.

One operational note for the host: cloud-init fetches the package with curl and no credentials, so r/vole has to be readable by anonymous visitors — vole admin acl set-default-role vole read. A private repository answers 404, which surfaces as “the server is up but the forge is not” in the troubleshooting section.

keenan 3d ago

Removed the container deployment too. Head is 9bce372; the change is docs plus deletions, nothing functional.

The argument that settled it: a Vole instance is one process whose state is a directory on local disk, so a container adds a layer without removing one — and it costs something real. The service unit’s hardening disappears, and the CI half needs an OCI runtime the container only reaches through the host’s socket, which is the thing docs/runner.md refuses. It was also the weakest path in practice: no image is published, so an upgrade meant a source checkout and a Go build where the package is one apt install, and CI never ran the container at all.

Gone: deploy/docker/ (Dockerfile, compose, entrypoint, image config), docs/deploy/docker.md, .dockerignore, the Docker row in the index, check_container() and the docker/entrypoint.sh check in check-deploy.py.

Kept, deliberately: every other mention of Docker in the repository, because those mean the CI container runtime — README.md, docs/workflows.md, docs/runner.md, .vole/images/*, the --container-runtime flag. The only remaining docker string in the tree is Prism’s docker.js grammar.

Where users who manage machines with Compose or Kubernetes now land: the index says there is no image and why, and tells them to give the forge a small VM of its own, where the first-boot file works unchanged. deploy/README.md keeps the same reasoning for contributors.

The deployment surface is now three files and a renderer:

deploy/cloud-init/vole.yaml      the forge
deploy/cloud-init/runner.yaml    a CI runner
deploy/cloud-init/render.sh      fills either one in
deploy/README.md

vole workflow run verify passes on the branch. python3 scripts/check-deploy.py no longer checks compose or Dockerfile paths; it checks the two templates, their boot scripts, and the runner’s unit/env/template agreement.