vole
Clone
Read only, no credentials:
https://rodent.software/git/vole.git
Write over SSH with a registered key:
git clone ssh://git@0.0.0.0:2222/vole.git
A free and open-source forge that lives in your Git repo.
Docs: rewrite the README and organize the guides
520470e19cd906bd6fff987e2d132d793ce56486 · Keenan Nemetz · 16d ago · parent c5902451
Docs: rewrite the README and organize the guides Replace the long README with a landing page, split feature details into focused guides, and remove the roadmap and spike notes. Update comments that referenced them.
README.md +49−703
| @@ -1,735 +1,81 @@ | |||
| 1 | 1 | # Vole | |
| 2 | 2 | ||
| 3 | A small but powerful software forge built on Git. | ||
| 3 | **A small but mighty software forge that lives in your Git repo.** | ||
| 4 | 4 | ||
| 5 | ## About | ||
| 5 | Vole is for developers who value ownership, simplicity, and resilience over feature | ||
| 6 | count. Code, issues, change requests, approvals, releases, and CI records live in | ||
| 7 | Git. Runners run where you put them. There is no control plane in a data center | ||
| 8 | you do not own. | ||
| 6 | 9 | ||
| 7 | Vole turns a git repo into a fully functional software forge. | ||
| 10 | ## Why Vole | ||
| 8 | 11 | ||
| 9 | - CI / CD workflows that run anywhere | ||
| 10 | - Replicated issues, change requests, and ACLs stored as git objects | ||
| 11 | - Integrated LFS support for release artifacts and CI / CD logs | ||
| 12 | - Every repo is a hosted website for landing pages and docs | ||
| 13 | - One auditable path: keys authenticate every push, one gate decides, one log records what | ||
| 14 | it decided | ||
| 12 | - **Your project survives outages.** Issues, reviews, approvals, releases, and | ||
| 13 | build records clone with the code, so they are not trapped in a vendor's | ||
| 14 | database. When a hosting service goes down, your project does not. | ||
| 15 | - **CI is not tied to a vendor.** Add a workflow, start a runner anywhere, and Vole | ||
| 16 | sends it work over SSH. Your build machines can be a server, a spare desktop, or | ||
| 17 | a box in the closet. | ||
| 18 | - **It is easy to run.** One binary, one config file, and no extra services to | ||
| 19 | operate. You do not need a cloud account to get started. | ||
| 20 | - **It is small on purpose.** Fewer moving parts, fewer things to break, and a | ||
| 21 | codebase you can read. Vole focuses on the work most teams actually do. | ||
| 22 | - **It is open source.** The whole thing is yours to read, run, and change. | ||
| 15 | 23 | ||
| 16 | ## Roadmap | ||
| 24 | ## What Vole is not | ||
| 17 | 25 | ||
| 18 | See [docs/roadmap.md](docs/roadmap.md) for the phased implementation plan. | ||
| 26 | Vole is deliberately smaller than GitHub or GitLab. It does not try to be everything to | ||
| 27 | every team. If you need a large feature matrix or enterprise controls, another | ||
| 28 | forge is probably a better fit. | ||
| 19 | 29 | ||
| 20 | ## Status | ||
| 21 | |||
| 22 | **Phases 1–6 are implemented.** Vole is a working forge: writes go over SSH and are | ||
| 23 | authenticated by your key; reads are open. | ||
| 24 | |||
| 25 | Working today: | ||
| 26 | |||
| 27 | - `git push` / `fetch` / `clone` over SSH, authenticated by a registered SSH key. | ||
| 28 | There is no password, token, or session anywhere in the system | ||
| 29 | - A read-only web interface for public repositories, with no login | ||
| 30 | - Anonymous HTTP clone for public repositories (useful for CI, or a network that | ||
| 31 | blocks port 22) | ||
| 32 | - Per-repo roles (read / write / admin) stored in the instance database beside the | ||
| 33 | roster, keyed by account id, so a rename or a re-import does not disturb them and | ||
| 34 | the one backup that holds the accounts holds the rules that name them | ||
| 35 | - Policy enforced in `pre-receive`, so a rejected push moves no refs at all | ||
| 36 | - Protected refs, per-blob size limits, and a push cap enforced by git's own | ||
| 37 | `receive.maxInputSize` | ||
| 38 | - The user roster lives in one SQLite database, `data/users.db`: an account is a | ||
| 39 | stable id plus a mutable name, with its SSH public keys, and a retired name is | ||
| 40 | never issued twice. The same file holds each repository's access rules, keyed by | ||
| 41 | account id — they are names for accounts, so they belong with the accounts. Nothing stored is secret. A running instance re-reads the | ||
| 42 | database every few seconds, so `vole user add-key` or `vole user retire` applies | ||
| 43 | without a restart and revocation takes effect the same way; an append-only | ||
| 44 | `data/audit.log` records every change. A clone has no copy of the roster — the | ||
| 45 | CLI reads the id/name map it needs over the same SSH key it pushes with | ||
| 46 | - Issues and change requests stored as files in the repository on the `vole` | ||
| 47 | branch, validated at push time and rendered read-only on the web | ||
| 48 | - Review as a git artifact: an approval names the exact commit it reviewed, and | ||
| 49 | `git push` is the merge, gated by `pre-receive`. The web shows the review: a commit | ||
| 50 | page with its patch, and a change request with the diff that would land | ||
| 51 | - CI: workflows in `.vole/workflows/*.yaml`, validated by the push that introduces | ||
| 52 | them. A push queues jobs; `vole runner` leases one over SSH with the same key, runs | ||
| 53 | the shell steps, uploads the attempt's log and commits its record. A job can declare `artifacts:` | ||
| 54 | (path patterns), which the runner uploads and the run page serves. Protected | ||
| 55 | branches can require green checks as well as approvals. One runner pool serves | ||
| 56 | every repository on the instance, an idle runner parks on the queue instead of | ||
| 57 | polling it, a newer push cancels the queued builds of the one it replaced, and | ||
| 58 | `/runners` shows who is connected, what each is building and how much is waiting | ||
| 59 | - LFS: large files stay out of the git object database and live in one | ||
| 60 | content-addressed store per instance. Uploads ride the SSH key | ||
| 61 | (`git-lfs-transfer`); downloads come through the HTTP clone URL, anonymously for a | ||
| 62 | public repository. A pointer whose object is missing fails the push that introduced | ||
| 63 | it, and `vole gc` sweeps what nothing reaches | ||
| 64 | - Releases are tags: the tag message is the notes, `.tar` / `.tar.gz` are generated from | ||
| 65 | the tagged tree per request (with an `ETag` from the tag object and a `304` for repeat | ||
| 66 | downloads), and binaries travel as LFS pointers on the `vole` branch — `vole release | ||
| 67 | attach` uploads them over the same SSH key and the store keeps one copy of identical | ||
| 68 | bytes. `vole release create` writes the annotated tag; `git tag -a` does the same thing | ||
| 69 | without the CLI | ||
| 70 | - Repo create / rename / delete from the CLI | ||
| 71 | - A derived SQLite index that can be deleted and rebuilt from the repos | ||
| 72 | - Static sites: a repository's `pages` branch is served as a website at | ||
| 73 | `<repo>.<pages_domain>`, straight out of git — no export step, ETags from blob | ||
| 74 | shas, `304`s for unchanged pages. `vole pages deploy <dir>` publishes a build | ||
| 75 | directory as a snapshot, `status` reports what is live, `unpublish` takes it down | ||
| 76 | - One secret on disk: the SSH host key. Nothing the instance writes is signed, because a | ||
| 77 | statement only the box can check is worth nothing to anyone else | ||
| 78 | |||
| 79 | Not yet: retention for run artifacts (they live as long as the run does). | ||
| 80 | |||
| 81 | ### The trust model | ||
| 82 | |||
| 83 | One model, everywhere: **identity is established cryptographically at the SSH handshake | ||
| 84 | and resolved to an account by the roster; every decision about that account is made at one | ||
| 85 | boundary (`pre-receive`) and written to one append-only log (`data/audit.log`).** Nothing | ||
| 86 | else in the system claims to prove anything. | ||
| 87 | |||
| 88 | So `git commit -S` is not required and not checked: a signature would restate the | ||
| 89 | handshake, it cannot bind a signature to an author (git's verifier reports a principal, not | ||
| 90 | a committer), and making it mandatory would tax every contributor to authenticate the least | ||
| 91 | contentious artefact in the system. The same goes for approvals, comments and CI results — | ||
| 92 | each is attributed by the account whose key pushed it. | ||
| 93 | |||
| 94 | What that buys is coherence rather than offline proof: a clone of a repository cannot, by | ||
| 95 | itself, show who pushed what. What the instance owes in exchange is a complete record, so | ||
| 96 | every accepted push is logged with its actor, and so is every privileged act — | ||
| 97 | force-pushes, lock overrides, release rewrites, merges through review and roster changes. | ||
| 98 | The trade is written down in the roadmap, and it is the wrong one for an instance that gets | ||
| 99 | mirrored somewhere its operator does not control. | ||
| 100 | |||
| 101 | ## Try it | ||
| 30 | ## Quick start | ||
| 102 | 31 | ||
| 103 | 32 | ```sh | |
| 104 | 33 | go build -o vole ./cmd/vole | |
| 105 | 34 | ||
| 106 | ./vole init --dir . # config + data/users.db | ||
| 35 | ./vole init --dir . # create config and data/users.db | ||
| 107 | 36 | ./vole user create you | |
| 108 | 37 | ./vole user add-key you --key ~/.ssh/id_ed25519.pub | |
| 109 | 38 | ./vole repo create demo --owner you | |
| 110 | ./vole acl default demo read # publish: 404 until you do this | ||
| 111 | ./vole serve # ssh :2222, http :8080 | ||
| 39 | ./vole acl default demo read # publish: the repo is private until you do this | ||
| 40 | ./vole serve # SSH :2222, web :8080 | ||
| 112 | 41 | ``` | |
| 113 | 42 | ||
| 114 | Then push with your key, and read without one: | ||
| 43 | Write with your key. Read without one: | ||
| 115 | 44 | ||
| 116 | 45 | ```sh | |
| 117 | 46 | git clone ssh://git@127.0.0.1:2222/demo.git # read/write, key required | |
| 118 | 47 | git clone http://127.0.0.1:8080/git/demo.git # read only, no credentials | |
| 119 | 48 | ``` | |
| 120 | 49 | ||
| 121 | Repositories are private by default. A private repo is a `404` on the web and | ||
| 122 | over HTTP — not a `403` — so the read-only surface never confirms that it exists. | ||
| 123 | |||
| 124 | Nothing in the configuration knows where the instance is reachable from outside, | ||
| 125 | so tell it once if the web is served through a proxy or another address: | ||
| 126 | |||
| 127 | ```toml | ||
| 128 | public_url = "https://forge.example.com" # optional; the CLI prints links to it | ||
| 129 | ``` | ||
| 130 | |||
| 131 | ## Extending Vole | ||
| 132 | |||
| 133 | The forge is also a Go library. The public packages are grouped by | ||
| 134 | responsibility, and a tool can talk to an instance or embed one without shelling | ||
| 135 | out to `vole`: | ||
| 136 | |||
| 137 | - `client` — the clone side: metadata branch writes, the roster and `whoami` | ||
| 138 | SSH verbs, object transfer, review helpers, and site deploys | ||
| 139 | - `core/id`, `core/git`, `core/lfs` — shared primitives and storage | ||
| 140 | - `forge` — instance lifecycle and administration; `forge/config`, | ||
| 141 | `forge/identity`, `forge/access`, `forge/repo`, and `forge/index` are its stores | ||
| 142 | - `metadata` — everything on `refs/heads/vole`: issues, requests, releases, | ||
| 143 | comments, approvals, and their validation | ||
| 144 | - `ci` — workflow validation and the queue (`ci/runs`); `ci/agent` is the | ||
| 145 | embeddable CI agent, used as `agent.Run` | ||
| 146 | - `server` — listeners and wiring; `server/ssh`, `server/githttp`, | ||
| 147 | `server/web`, `server/pages`, and `server/hook` are the edge adapters | ||
| 148 | |||
| 149 | `cmd/vole` stays a CLI: it parses git-style flags, chooses a command, and prints | ||
| 150 | the result. Logic that more than one caller needs lives in the packages above. | ||
| 151 | |||
| 152 | ## Filing an issue | ||
| 153 | |||
| 154 | An issue is one file on the `vole` branch. No client tooling is required: | ||
| 155 | |||
| 156 | ```sh | ||
| 157 | git switch --orphan vole | ||
| 158 | ID=01M5JR2JK29KKMAA1T9MDNTPBP # any 26-char ULID; unique per issue | ||
| 159 | mkdir -p issues/$ID | ||
| 160 | cat > issues/$ID/issue.md <<'MD' | ||
| 161 | --- | ||
| 162 | title: "Nothing renders on the docs page" | ||
| 163 | state: open | ||
| 164 | author: you | ||
| 165 | created: 2026-10-22T21:30:00Z | ||
| 166 | labels: | ||
| 167 | - bug | ||
| 168 | --- | ||
| 169 | Steps to reproduce... | ||
| 170 | MD | ||
| 171 | git add -A && git commit -m "issue: docs page blank" && git push origin vole | ||
| 172 | ``` | ||
| 173 | |||
| 174 | The head is frontmatter, the rest is the body. That shape is the reason the | ||
| 175 | format exists: closing an issue changes one line, adding a label adds one line, | ||
| 176 | and `git blame` still attributes the title to whoever wrote it. Free text | ||
| 177 | (`title`) is always double-quoted, so a `#` or a leading `-` cannot change what | ||
| 178 | it means; a title typed without quotes that YAML would read differently is | ||
| 179 | refused by name rather than stored as something shorter. | ||
| 180 | |||
| 181 | That branch holds Vole's metadata and is deliberately narrow: | ||
| 182 | |||
| 183 | - Only `issues/` and `requests/` may appear in it; anything else is rejected at | ||
| 184 | push. | ||
| 185 | - History must be linear. Merge commits are refused, so `git log vole` reads as | ||
| 186 | a list of changes rather than a graph. | ||
| 187 | - Force-pushing it requires repo admin and is written to `data/audit.log`, because | ||
| 188 | rewriting history is a redaction tool, not a normal operation. | ||
| 189 | - If someone else pushes first, `vole issue new` re-applies your change on the new | ||
| 190 | tip and retries; you do not have to resolve anything by hand. | ||
| 191 | |||
| 192 | The id must be a ULID (26 characters, Crockford base32 — uppercase, no `I`, `L`, | ||
| 193 | `O` or `U`). A UUID is not one and will be rejected. | ||
| 194 | |||
| 195 | Or use the CLI, which is the same thing with the paperwork done: | ||
| 196 | |||
| 197 | ```sh | ||
| 198 | vole whoami # which account your key is; no guessing | ||
| 199 | vole issue new --title "Nothing renders on the docs page" -m "Steps..." -l bug -a wilbur | ||
| 200 | vole issue comment -m "Also affects search" 01M5JR2JK29KKMAA1T9MDNTPBP # prints the comment id | ||
| 201 | vole issue edit-comment 01M5JR2JK29KKMAA1T9MDNTPBP 01M5N1PRC4GJPZNEEG8PR8J4D6 -m "Corrected." | ||
| 202 | vole issue rm-comment 01M5JR2JK29KKMAA1T9MDNTPBP 01M5N1PRC4GJPZNEEG8PR8J4D6 | ||
| 203 | vole issue edit 01M5JR2JK29KKMAA1T9MDNTPBP --title "A better title" -l bug -l docs | ||
| 204 | vole issue assign 01M5JR2JK29KKMAA1T9MDNTPBP wilbur | ||
| 205 | vole issue unassign 01M5JR2JK29KKMAA1T9MDNTPBP wilbur | ||
| 206 | vole issue close 01M5JR2JK29KKMAA1T9MDNTPBP | ||
| 207 | vole issue list --state open -l bug --assignee wilbur --format '%(shortid) %(title)' | ||
| 208 | |||
| 209 | vole issue show 01M5JR2JK29KKMAA1T9MDNTPBP # a prefix is enough | ||
| 210 | ``` | ||
| 211 | |||
| 212 | An assignee is a roster account name, so it is held to the same rule an account | ||
| 213 | name is (lowercase letters, digits, `-` and `_`) and a value that could never be | ||
| 214 | one fails the push that introduced it. The name is stored as written rather than | ||
| 215 | resolved to an account id: names are never reissued, so an assignment keeps | ||
| 216 | meaning the same person after a rename, and the file stays readable in a plain | ||
| 217 | clone. | ||
| 218 | |||
| 219 | Options follow git's conventions: `--long value`, `--long=value`, `-o value`, | ||
| 220 | `-ovalue`, repeatable `-l`, `--` to end option parsing, and `-h` for usage. | ||
| 221 | Options may come before or after arguments, as in git itself — both | ||
| 222 | `vole issue comment -m text <id>` and `vole issue comment <id> -m text` work — and a | ||
| 223 | usage mistake exits 129, as git does. | ||
| 224 | |||
| 225 | The CLI writes the files, commits to the `issues` branch, validates locally and | ||
| 226 | pushes. Its author is the account your key resolves to — the CLI asks the instance | ||
| 227 | once (`vole-whoami`) instead of trusting `git config user.name`, which on a normal | ||
| 228 | machine is a real name that is not an account name. There is no `--author` flag to | ||
| 229 | get wrong. | ||
| 230 | |||
| 231 | A repository with no configured remote works too: `git init` in an empty directory | ||
| 232 | and pass the URL. | ||
| 233 | |||
| 234 | ```sh | ||
| 235 | git init bug-report && cd bug-report | ||
| 236 | vole issue new --title "the docs page is blank" --remote ssh://git@forge.example.com/demo.git | ||
| 237 | ``` | ||
| 238 | |||
| 239 | `#<ulid>` in a body links to that issue, and commits whose message mentions an id | ||
| 240 | are listed as backlinks on the issue. The list page filters by state, label and | ||
| 241 | assignee, and searches titles and bodies. When `public_url` is set, `new` and | ||
| 242 | `comment` print the page they wrote, and the comment link lands on that comment. | ||
| 243 | |||
| 244 | A comment is `issues/<id>/comments/<ulid>.md` with `author` and `created` | ||
| 245 | frontmatter. Closing an issue is editing `state` to `closed`. The push is | ||
| 246 | rejected with the offending path named if the tree does not match the format, and | ||
| 247 | every issue is a normal commit you can `git log`, `git blame` and merge. | ||
| 248 | |||
| 249 | Editing is a commit, not a special operation. Rewriting a comment's body keeps | ||
| 250 | its file, so `git log -p -- issues/<id>/comments/<ulid>.md` is what it used to | ||
| 251 | say; deleting one removes the file, and the text stays in history. Neither adds | ||
| 252 | an `edited` field — the commit that changed the file carries the time. | ||
| 253 | |||
| 254 | A comment belongs to whoever wrote it: `pre-receive` compares the author recorded | ||
| 255 | on the **old** blob (the one that was validated when it landed) against the | ||
| 256 | account the SSH key resolved to, so only that person can change or delete it, and | ||
| 257 | editing the author line in the same push does not help. A repo admin may override | ||
| 258 | — a thread written by someone who has left is otherwise unrepairable — and that | ||
| 259 | override is written to `data/audit.log`. The issue's own file is not covered: | ||
| 260 | closing, relabelling and assigning somebody else's issue is triage, not | ||
| 261 | authorship. Deleting is not redaction; redaction is still an admin force-push. | ||
| 262 | |||
| 263 | ## Reviewing a change | ||
| 264 | |||
| 265 | Review is a git artifact, not a web session. An approval records the exact commit | ||
| 266 | that was reviewed, so a branch that moves after approval is not approved any more. | ||
| 267 | Because `git push` is the merge, the gate lives in `pre-receive` and there is no | ||
| 268 | write surface to authenticate: | ||
| 269 | |||
| 270 | ```sh | ||
| 271 | # the author pushes a branch and opens a request | ||
| 272 | git push origin feature:refs/heads/feature | ||
| 273 | vole request new --title "Retry on timeout" --base main --head feature -m "Rationale." | ||
| 274 | |||
| 275 | # an approver reviews that commit and approves it | ||
| 276 | vole request approve <id> --as <your-name> | ||
| 277 | |||
| 278 | # the author lands it; the hook checks the approvals match the new tip | ||
| 279 | git push origin feature:refs/heads/main | ||
| 280 | ``` | ||
| 281 | |||
| 282 | The request page shows what would land — the diff against the merge base, the CI | ||
| 283 | checks for the head, how many approvals are required and who has approved which | ||
| 284 | commit — so a review does not start with a clone. It also answers the one question a | ||
| 285 | diff cannot: whether the branch still **merges** ("this branch no longer merges into | ||
| 286 | main cleanly: `code.txt`"), computed with `git merge-tree` in memory, so a branch that | ||
| 287 | needs a rebase is visible before anybody approves it. That answer is shown, never | ||
| 288 | enforced, and it is left out entirely when the question cannot be asked — a base | ||
| 289 | branch that no longer exists, or a head that was never pushed. The push is refused | ||
| 290 | unless the new tip *is* the approved commit, or is a | ||
| 291 | merge whose parents are the branch tip and the approved commit. That second shape is why | ||
| 292 | "merge, then push" works while "append a commit and call it approved" does not. | ||
| 293 | |||
| 294 | `vole request show <id>` prints the same `merge:` line from a clone, and says so when | ||
| 295 | the head is not in the clone rather than guessing. | ||
| 296 | |||
| 297 | A request can be discussed, and the discussion is a normal pushed file — the same | ||
| 298 | shape as a comment on an issue: | ||
| 299 | |||
| 300 | ```sh | ||
| 301 | vole request comment <id> -m "Does this hold for the empty repository case?" | ||
| 302 | vole request comment <id> -m "Good catch, amended." # prints the comment id | ||
| 303 | vole request edit-comment <id> <comment-id> -m "Rephrased." | ||
| 304 | vole request rm-comment <id> <comment-id> | ||
| 305 | ``` | ||
| 306 | |||
| 307 | A comment belongs to whoever wrote it: only that account can change or delete it, | ||
| 308 | and an admin override is audited. The thread shows on the request page with an | ||
| 309 | anchor per comment (`#c-<ulid>`), so a remark can be linked. | ||
| 310 | |||
| 311 | A comment can also point at a line, on either side of the diff: | ||
| 50 | New repositories are private. If someone opens a private repository on the web, | ||
| 51 | they get a `404` — the same answer as a repository that does not exist — so Vole | ||
| 52 | never confirms that it is there. | ||
| 312 | 53 | ||
| 313 | ```sh | ||
| 314 | vole request comment <id> -m "this drops the check" --path internal/auth.go --line 88 | ||
| 315 | vole request comment <id> -m "and this was the old line" --path internal/auth.go --line 88 --side old | ||
| 316 | ``` | ||
| 54 | ## Learn more | ||
| 317 | 55 | ||
| 318 | The anchor records the *content* that was reviewed (the blob id), not the line | ||
| 319 | number, so the comment stays attached to what it was written about: the request page | ||
| 320 | marks the line while the file is unchanged, and marks the comment "the file has | ||
| 321 | changed since this comment" once it moves. The CLI needs the commits in the clone — | ||
| 322 | it prints `git fetch` rather than writing an anchor it cannot compute, and it | ||
| 323 | refuses a line past the end of the file. | ||
| 56 | | If you want to… | Start here | | ||
| 57 | |---|---| | ||
| 58 | | Set up your first repository | [Getting started](docs/getting-started.md) | | ||
| 59 | | Change bind addresses, limits, or proxy settings | [Configuration](docs/configuration.md) | | ||
| 60 | | Add people and choose who can do what | [Users and access](docs/users-and-access.md) | | ||
| 61 | | Keep issues with the code | [Issues](docs/issues.md) | | ||
| 62 | | Review and merge changes | [Change requests](docs/change-requests.md) | | ||
| 63 | | Run builds on your own machines | [CI](docs/ci.md) | | ||
| 64 | | Store large files or lock editable files | [LFS](docs/lfs.md) | | ||
| 65 | | Publish a release | [Releases](docs/releases.md) | | ||
| 66 | | Host a website from a repository | [Sites](docs/sites.md) | | ||
| 67 | | Back up or recover an instance | [Operations](docs/operations.md) | | ||
| 324 | 68 | ||
| 325 | ```sh | ||
| 326 | vole acl protect demo refs/heads/main # protect a branch | ||
| 327 | vole acl approvals demo 2 # require two approvals (default 1) | ||
| 328 | vole acl approvals demo 0 # back to admin-only | ||
| 329 | ``` | ||
| 69 | Reference: [CLI commands](docs/cli.md) · [Architecture](docs/architecture.md) | ||
| 330 | 70 | ||
| 331 | Admins can push a protected branch directly. Every merge is recorded in | ||
| 332 | `data/audit.log` with the request id and approvers. | ||
| 71 | Full index: [docs/README.md](docs/README.md). | ||
| 333 | 72 | ||
| 334 | An approval can only be written by the person it names: the hook compares the path | ||
| 335 | to the SSH identity the server resolved from your key, so `approvals/keenan.json` | ||
| 336 | cannot be created *or edited* by anyone else, and only keenan can withdraw it (an | ||
| 337 | admin can, and that is audited). The names in the frontmatter — `author` on an | ||
| 338 | issue or request — are display fields and carry no authority; the approval path is | ||
| 339 | the only name that decides who may land a commit. | ||
| 340 | |||
| 341 | ### Locking | ||
| 342 | |||
| 343 | A file git cannot merge — art, audio, a model, a level — can be reserved before somebody | ||
| 344 | edits it. `git lfs lock` works over the same SSH connection as everything else, and while | ||
| 345 | a path is held, a push that changes it is refused: | ||
| 73 | ## Development | ||
| 346 | 74 | ||
| 347 | 75 | ```sh | |
| 348 | git lfs lock art/hero.psd # now nobody else's push can change that path | ||
| 349 | git lfs unlock art/hero.psd # released | ||
| 350 | git lfs locks # who is holding what | ||
| 76 | go test ./... | ||
| 77 | ./scripts/ci.sh # everything CI runs, before you push | ||
| 351 | 78 | ``` | |
| 352 | 79 | ||
| 353 | The refusal is enforced by the server, not by the client's pre-push hook, so it holds for | ||
| 354 | a plain `git push` too. An admin can push past a lock, and that override is written to | ||
| 355 | `data/audit.log` — otherwise a lock whose holder has left would make a path unpushable | ||
| 356 | forever. `/r/<repo>/locks` shows what is held. | ||
| 357 | |||
| 358 | Locks are not part of a clone: they live beside the roster in `data/users.db`, because | ||
| 359 | they are ephemeral state that must not be resurrected from history. | ||
| 360 | |||
| 361 | ## LFS | ||
| 362 | |||
| 363 | Large files live in a content-addressed store outside git, one instance-wide pool with an | ||
| 364 | object stored once however many repositories refer to it. Objects move over the SSH | ||
| 365 | connection that carries the push, so there is no token and no second endpoint: | ||
| 366 | |||
| 367 | ```sh | ||
| 368 | git lfs install && git lfs track "*.psd" | ||
| 369 | git add .gitattributes art/hero.psd && git commit -m "add the hero art" && git push | ||
| 370 | ``` | ||
| 371 | |||
| 372 | An object is only fetchable through a repository that actually holds it: knowing a sha256 | ||
| 373 | is not enough, because each repository has its own view of the store. `vole gc` reconciles | ||
| 374 | those views from the refs and sweeps what nothing refers to, so it is also the repair | ||
| 375 | command — and an instance that stored objects before this existed links them on first | ||
| 376 | boot. | ||
| 377 | |||
| 378 | For files git cannot merge, see [Locking](#locking). | ||
| 379 | |||
| 380 | ## CI | ||
| 381 | |||
| 382 | Workflows live in `.vole/workflows/*.yaml`, are validated by the push that introduces | ||
| 383 | them, and run on a runner you start anywhere: | ||
| 384 | |||
| 385 | ```yaml | ||
| 386 | version: 1 | ||
| 387 | on: | ||
| 388 | push: | ||
| 389 | branches: [main] | ||
| 390 | jobs: | ||
| 391 | build: | ||
| 392 | runs-on: [linux, arm64] # only a runner carrying both labels takes this job | ||
| 393 | steps: | ||
| 394 | - name: test | ||
| 395 | run: go test ./... | ||
| 396 | - name: build | ||
| 397 | run: make dist | ||
| 398 | artifacts: | ||
| 399 | - path: dist/* | ||
| 400 | pages: | ||
| 401 | path: site/ # publish this directory as the repository's site | ||
| 402 | ``` | ||
| 403 | |||
| 404 | ```sh | ||
| 405 | vole runner --remote ssh://git@host:2222 # long-poll; --once runs one job | ||
| 406 | ``` | ||
| 407 | |||
| 408 | A push that touches a workflow queues one run per job, pinned to the commit *and* to | ||
| 409 | the workflow file's blob, so editing the workflow afterwards cannot change what a | ||
| 410 | queued run executes. `on.push.branches` filters branch pushes and `on.push.tags` | ||
| 411 | filters tag pushes: a workflow with neither runs on every branch, and one that says | ||
| 412 | `tags:` runs *only* on tags, because naming your trigger is naming it. | ||
| 413 | |||
| 414 | The queue follows the refs, not just the push notification. A push is delivered by the | ||
| 415 | hook in well under a second, and every 30 seconds the server also compares each branch | ||
| 416 | and tag tip against the last tip it queued — so a push whose notification was lost (a | ||
| 417 | crash between the ref update and the hook's write) still runs. Deleting `data/runs.db` | ||
| 418 | loses the attempts that were in flight: the queue is rebuilt from the refs (each | ||
| 419 | current tip is queued once, since the notes about what had been queued went with the | ||
| 420 | file), and `vole index rebuild` brings the finished history back from the records on | ||
| 421 | each repository's `vole` branch. Pushing again cancels the *queued* runs of the commit | ||
| 422 | it replaced — never a running one — and one runner pool serves every repository: an | ||
| 423 | idle runner parks on the lease instead of asking again, and the lease prefers the | ||
| 424 | repository with the least work in flight. `max_jobs_per_repo` bounds how much of the | ||
| 425 | pool one repository may occupy, when ordering is not enough. | ||
| 426 | |||
| 427 | `runs-on` names the machine a job needs, as one label or a list, and a job goes only to | ||
| 428 | a runner that carries *every* label it asks for (extra labels on the runner are fine). | ||
| 429 | A job with no `runs-on` runs anywhere, which is what every job did before targeting | ||
| 430 | existed. The labels are a claim by the runner about itself, not a permission: | ||
| 431 | |||
| 432 | ```sh | ||
| 433 | vole runner --remote ssh://git@host:2222 --label linux --label arm64 | ||
| 434 | vole runner --remote ssh://git@host:2222 --repos demo --repos docs # only these | ||
| 435 | ``` | ||
| 436 | |||
| 437 | `--repos` is the other direction and is *not* a claim: it narrows what the account's | ||
| 438 | runner allowlist already grants, and the server applies the intersection, so the worst a | ||
| 439 | runner can do is take less work than it was allowed. A filter that matches nothing is | ||
| 440 | refused at startup rather than presenting as a healthy idle runner. | ||
| 441 | |||
| 442 | `/runners` is the page for "why has my build not started": the connected runners and | ||
| 443 | what each takes, the job each is building (or that it is waiting for one), and how much | ||
| 444 | each repository has queued and running — including whether it is at its | ||
| 445 | `max_jobs_per_repo` cap. It follows the same visibility rule as everything else, so a | ||
| 446 | runner building a private repository is not shown. | ||
| 447 | |||
| 448 | ### Releases built by CI | ||
| 449 | |||
| 450 | A tag push is a release build. Mark the artifacts `release: true` and the run commits | ||
| 451 | them as that tag's release artifacts — the same pointers `vole release attach` writes, | ||
| 452 | so the release page, the ETags and the immutability rules are unchanged: | ||
| 453 | |||
| 454 | ```yaml | ||
| 455 | version: 1 | ||
| 456 | on: | ||
| 457 | push: | ||
| 458 | tags: ['v*'] | ||
| 459 | jobs: | ||
| 460 | release: | ||
| 461 | steps: | ||
| 462 | - name: build | ||
| 463 | run: make dist | ||
| 464 | artifacts: | ||
| 465 | - path: dist/* | ||
| 466 | release: true | ||
| 467 | ``` | ||
| 468 | |||
| 469 | ```sh | ||
| 470 | git tag -a v1.0.0 -m "first release" # a release needs notes and a tagger | ||
| 471 | git push origin refs/tags/v1.0.0 # the runner builds and publishes | ||
| 472 | ``` | ||
| 473 | |||
| 474 | The metadata commit names the build (`release: v1.0.0 artifacts from run 01M5...`), so | ||
| 475 | `git log vole` says which run shipped the bytes. A lightweight tag is refused at the | ||
| 476 | publish — `git tag -a`, not `git tag` — because a tag with no message has no release | ||
| 477 | notes and no tagger. A run for a tag never counts as a branch's required check, even | ||
| 478 | when the tag points at that branch's tip. The runner leases a job over the same SSH key you push with, | ||
| 479 | checks out the commit, runs the steps with per-step timeouts, and uploads the | ||
| 480 | attempt's log to the object store before reporting the exit code and the log's | ||
| 481 | object id. Nothing is streamed: a log the server assembled would be a log the | ||
| 482 | server wrote, and the runner is the party that can be held to it. `/r/<repo>/runs/<id>/log` | ||
| 483 | serves it, `?attempt=N` picks one attempt out of a retried job, and an attempt whose | ||
| 484 | runner was killed has no log at all. | ||
| 485 | |||
| 486 | A runner publishes every result it reports. When an attempt ends the runner commits | ||
| 487 | `runs/<run id>/<attempt>/` — the record (repo, ref, SHA, the workflow blob, the digest of | ||
| 488 | the pinned steps, the exit code, the finish time, who reported it and the machine) plus a | ||
| 489 | pointer to the log and one per artifact — together with any release pointers the build | ||
| 490 | published, and pushes that once. The push is what `pre-receive` checks: the account must | ||
| 491 | be on the repository's runner allowlist (`vole acl runners`), the record must name that | ||
| 492 | account, and it must describe the job that account was leased. The run page names the | ||
| 493 | account and links the commit that carries the record. | ||
| 494 | |||
| 495 | So the record is not a signed statement, and it does not need to be: attribution in this | ||
| 496 | forge is established at the push boundary, once, the same way an issue's author or an | ||
| 497 | approval's owner is — see [the trust model](#the-trust-model). What the instance owes in | ||
| 498 | exchange is the record of its own decision, which the push log keeps. | ||
| 499 | |||
| 500 | A result counts only if the host allowed the account that reported it. | ||
| 501 | `vole acl runners <demo> ci-runner-1` names the accounts whose builds this repository | ||
| 502 | believes. Names replace the list, `--add` and `--remove` change it a name at a time, | ||
| 503 | and `--clear` goes back to admins only. | ||
| 504 | Admins are implicit — they can edit the list, and a protected branch already lets them | ||
| 505 | land anything without checks — so the list is about everybody else: **a writer running | ||
| 506 | CI on their laptop does not turn a required check green.** Their result is still | ||
| 507 | recorded, and the run page says whether it counts. The same list decides who may lease | ||
| 508 | a job for the repository, since running a job executes its code. | ||
| 509 | |||
| 510 | A runner that disappears has its job requeued; after five attempts the job fails | ||
| 511 | instead of cycling forever. Protected branches can require green checks as well as | ||
| 512 | approvals. | ||
| 513 | |||
| 514 | `artifacts:` declares the files a run publishes. The patterns are expanded in the | ||
| 515 | checkout after the steps pass, uploaded over SSH into the same content-addressed | ||
| 516 | store that holds LFS objects and CI logs, and listed on the run page with a download | ||
| 517 | each: | ||
| 518 | |||
| 519 | ```sh | ||
| 520 | curl -OJ http://host/r/demo/runs/01M5.../artifacts/tool-linux | ||
| 521 | sha256sum tool-linux | ||
| 522 | ``` | ||
| 523 | |||
| 524 | A pattern that matches nothing **fails the job** rather than publishing nothing — a | ||
| 525 | build that stopped producing the file is exactly what the declaration is there to | ||
| 526 | catch. Two matches with the same base name are refused too, so nothing is published | ||
| 527 | under a name nobody chose. The bytes are kept while the run is in the queue: `vole gc` | ||
| 528 | marks them from the queue, since nothing in git names a build output. | ||
| 529 | |||
| 530 | An artifact of a run is not a release artifact. It belongs to the run that built it, | ||
| 531 | and publishing it to a release is a separate, deliberate step (`vole release attach`), | ||
| 532 | so "latest build" and "the bytes v1.0.0 ships" are never the same link. | ||
| 533 | |||
| 534 | `pages:` deploys a directory as the repository's site, with the same snapshot rules as | ||
| 535 | `vole pages deploy`: the branch's tree becomes exactly those files, so a page deleted | ||
| 536 | from the build stops being served. The commit names the build that produced it | ||
| 537 | (`site: main from run 01M5... (6beaa53)`), so `git log pages` in any clone says what | ||
| 538 | put that tree there. | ||
| 539 | |||
| 540 | A deploy is refused when the run is no longer the newest commit on its branch — an | ||
| 541 | older build finishing late would otherwise revert the site to an older tree, and a | ||
| 542 | silent revert is worse than a failed job. The run log says so: | ||
| 543 | `this run built f1f415ec but main is at 6beaa530 now; the site was not deployed`. | ||
| 544 | |||
| 545 | Only one job per workflow may publish the site: two would race, and the winner would be | ||
| 546 | whichever runner finished last. The branch name is instance configuration | ||
| 547 | (`pages_ref`), so it comes from the server rather than from the workflow. | ||
| 548 | |||
| 549 | ## Large files | ||
| 550 | |||
| 551 | Track a pattern with git-lfs as usual; the only thing that changes is where the bytes | ||
| 552 | go. Vole answers `git-lfs-transfer` on the SSH connection it already trusts, so there | ||
| 553 | is no token to mint and nothing extra to configure: | ||
| 554 | |||
| 555 | ```sh | ||
| 556 | git lfs install # once per machine | ||
| 557 | git lfs track "*.psd" | ||
| 558 | git add .gitattributes logo.psd | ||
| 559 | git commit -m "add the logo" && git push origin main | ||
| 560 | |||
| 561 | git clone http://host/git/demo.git # LFS objects come through the same URL | ||
| 562 | ``` | ||
| 563 | |||
| 564 | Repositories keep the pointer; Vole keeps the object, once, however many repositories | ||
| 565 | use it. Configure the sizes in `vole.toml`: | ||
| 566 | |||
| 567 | ```toml | ||
| 568 | lfs_object_max_bytes = 2147483648 # one object; larger is refused, not spooled | ||
| 569 | lfs_warn_bytes = 5242880 # a binary this big committed raw gets a warning | ||
| 570 | ``` | ||
| 571 | |||
| 572 | A push that lands a binary over `lfs_warn_bytes` without LFS is accepted with the | ||
| 573 | `git lfs track` command to fix it; a push that commits a pointer whose object was never | ||
| 574 | uploaded is refused, because nobody could check that commit out. Objects are swept | ||
| 575 | only when nothing reaches them: | ||
| 576 | |||
| 577 | ```sh | ||
| 578 | vole gc --dry-run # mark from every ref, report what is unreachable | ||
| 579 | vole gc # remove it (objects younger than --grace stay) | ||
| 580 | ``` | ||
| 581 | |||
| 582 | ## Releases | ||
| 583 | |||
| 584 | A release is an annotated tag: the message is the notes and the tree is the source | ||
| 585 | tarball, so both travel with any clone and there is no release table to drift. | ||
| 586 | |||
| 587 | ```sh | ||
| 588 | vole release create v1.0.0 -F notes.md | ||
| 589 | git push --follow-tags | ||
| 590 | |||
| 591 | # plain git does the same thing, which is the point | ||
| 592 | git tag -a v1.0.0 -F notes.md && git push --follow-tags | ||
| 593 | ``` | ||
| 594 | |||
| 595 | A workflow can build the release instead of a person attaching files by hand — see | ||
| 596 | [Releases built by CI](#releases-built-by-ci). | ||
| 597 | |||
| 598 | `/r/<repo>/releases` lists every tag and marks the lightweight ones, because a tag with | ||
| 599 | no tagger has no notes to show. The release page carries the notes, the commits since the | ||
| 600 | previous release, and two downloads: | ||
| 601 | |||
| 602 | ```sh | ||
| 603 | curl -O http://host/r/demo/releases/v1.0.0/archive.tar | ||
| 604 | curl -OJ http://host/r/demo/releases/v1.0.0/archive.tar.gz | ||
| 605 | ``` | ||
| 606 | |||
| 607 | Both are produced from the tag when you ask, so nothing can go stale: the `ETag` is the | ||
| 608 | tag object id, `If-None-Match` gets a `304`, and the same tag yields the same bytes on | ||
| 609 | every instance. Measured on an 11.6 MiB, 2,566-file repository: 54 ms for `.tar`, 218 ms | ||
| 610 | for `.tar.gz`, and every extracted file checks out against the tree. A repository that | ||
| 611 | tracks files with Git LFS gets a warning on the page, because `git archive` copies | ||
| 612 | pointers rather than their contents. | ||
| 613 | |||
| 614 | Binaries attach to the release the same way large files move everywhere else — over SSH, | ||
| 615 | into the content-addressed store, with a pointer committed for them: | ||
| 616 | |||
| 617 | ```sh | ||
| 618 | vole release attach v1.0.0 dist/vole-linux-amd64 dist/vole-darwin-arm64 | ||
| 619 | curl -O http://host/r/demo/releases/v1.0.0/artifacts/vole-linux-amd64 | ||
| 620 | sha256sum vole-linux-amd64 | ||
| 621 | ``` | ||
| 622 | |||
| 623 | Attaching byte-identical files twice stores one object and transfers nothing the second | ||
| 624 | time. Downloads carry the object id as `ETag` (so `304` works), `Content-Disposition: | ||
| 625 | attachment` with `nosniff` (an artifact never renders on the forge's origin), and Range | ||
| 626 | support, so a 100 MiB download resumes. The read rule is the clone rule: artifacts of a | ||
| 627 | private repository are `404`. Attach *after* pushing the tag, not before: the pointer is | ||
| 628 | what publishes the bytes, and a pointer whose object was never uploaded is refused by | ||
| 629 | `pre-receive` rather than advertised as a download. | ||
| 630 | |||
| 631 | Artifacts are reachable by `vole gc` because the pointers live on the `vole` branch: | ||
| 632 | |||
| 633 | ```sh | ||
| 634 | vole gc --dry-run # the published artifact is a reachable root | ||
| 635 | ``` | ||
| 636 | |||
| 637 | A release with artifacts is immutable from the moment they land: the server refuses to | ||
| 638 | move or delete that tag, to remove a pointer, or to re-point one at different bytes | ||
| 639 | (`pre-receive`), so a saved download keeps meaning what it said. Adding another artifact | ||
| 640 | is fine — that invalidates nothing. An admin can still repair a mistyped release, and the | ||
| 641 | override is written to `data/audit.log`. | ||
| 642 | |||
| 643 | ## Hosting a site | ||
| 644 | |||
| 645 | A repository's `pages` branch is a website at `<repo>.<pages_domain>`. Nothing is | ||
| 646 | built or exported: the push moves the branch, and the next request reads the new tree. | ||
| 647 | |||
| 648 | A site is usually a build directory, so the CLI publishes one as a snapshot — the | ||
| 649 | branch's tree becomes exactly those files, which is what makes a deleted page stop | ||
| 650 | being served: | ||
| 651 | |||
| 652 | ```sh | ||
| 653 | npm run build | ||
| 654 | vole pages deploy dist # publishes dist/ to the pages branch | ||
| 655 | vole pages deploy dist --dry-run # 12 added, 3 changed, 1 removed, nothing pushed | ||
| 656 | vole pages status # branch tip, commit, file count, total bytes | ||
| 657 | vole pages unpublish # site answers 404 within a second | ||
| 658 | ``` | ||
| 659 | |||
| 660 | Dotfiles are published (`.nojekyll`, `.well-known/`), `.git` is skipped, and symlinks | ||
| 661 | are skipped rather than followed — a site is bytes, and a link out of the directory | ||
| 662 | would publish something the author never chose. Deploying identical bytes twice pushes | ||
| 663 | nothing. `--branch` names a different branch, which is what you pass if the operator | ||
| 664 | changed `pages_ref`. | ||
| 665 | |||
| 666 | A workflow can deploy the site too (`pages:` in the CI section below), which is the | ||
| 667 | usual arrangement: the build produces `dist/`, and the deploy is the last thing the job | ||
| 668 | does. | ||
| 669 | |||
| 670 | Plain git is the same thing, for a hand-written page: | ||
| 671 | |||
| 672 | ```sh | ||
| 673 | git switch --orphan pages | ||
| 674 | printf '<h1>hello</h1>\n' > index.html | ||
| 675 | git add index.html && git commit -m "site" | ||
| 676 | git push origin pages # live at https://demo.pages.example.com/ | ||
| 677 | ``` | ||
| 678 | |||
| 679 | Turn it on in `vole.toml`: | ||
| 680 | |||
| 681 | ```toml | ||
| 682 | pages_domain = "pages.example.com" # <repo>.pages.example.com | ||
| 683 | pages_ref = "pages" # the branch to serve | ||
| 684 | pages_procs = 4 # cat-file processes per active site | ||
| 685 | pages_cache_bytes = 33554432 # 32 MiB of blobs, `(commit, path)` keyed | ||
| 686 | ``` | ||
| 687 | |||
| 688 | A directory serves its `index.html`; a directory URL without a trailing slash | ||
| 689 | redirects. Responses carry the blob sha as `ETag` and the commit time as | ||
| 690 | `Last-Modified`, so unchanged pages cost a `304`. A private repository has no site: | ||
| 691 | there is no session for an anonymous reader to authenticate with, so it answers `404` | ||
| 692 | exactly like its clone URL does. Visibility and the branch tip are re-read once a | ||
| 693 | second, so publishing or unpublishing a site takes effect within that window. | ||
| 694 | |||
| 695 | Sites are subdomains of the instance's own domain, so they ride the certificate you | ||
| 696 | already have. Terminate TLS in front of `http_addr` and make sure the proxy preserves | ||
| 697 | the `Host` header — `Host` is the only thing that says which site a request is for: | ||
| 698 | |||
| 699 | ```nginx | ||
| 700 | server { | ||
| 701 | listen 443 ssl; | ||
| 702 | server_name *.forge.example.com; # the forge and every site | ||
| 703 | location / { | ||
| 704 | proxy_set_header Host $host; # required: pages route on it | ||
| 705 | proxy_set_header X-Forwarded-Proto $scheme; | ||
| 706 | proxy_pass http://127.0.0.1:8080; | ||
| 707 | } | ||
| 708 | } | ||
| 709 | ``` | ||
| 710 | |||
| 711 | ## Backup | ||
| 712 | |||
| 713 | Three things, and all are small: | ||
| 714 | |||
| 715 | ```sh | ||
| 716 | ./vole user backup backup/users.db # accounts and their keys | ||
| 717 | cp data/ssh_host_ed25519_key backup/ # the only secret on the box | ||
| 718 | cp data/audit.log backup/ 2>/dev/null # who added what, when | ||
| 719 | ``` | ||
| 720 | |||
| 721 | `vole user backup` is a consistent copy taken through SQLite, so it is safe while | ||
| 722 | the instance is running; copying `users.db` by hand can miss rows the database has | ||
| 723 | not checkpointed yet. Losing the roster means re-adding keys, not losing | ||
| 724 | repositories — nothing in it is secret, so a copied file leaks nothing. | ||
| 725 | |||
| 726 | The host key is the only secret the instance holds. Nothing is signed: not results, not | ||
| 727 | approvals, not pushes. Attribution comes from the SSH handshake, enforcement from | ||
| 728 | `pre-receive`, and evidence from `data/audit.log` — one of each, deliberately, so there is | ||
| 729 | no second path that can drift from the first. | ||
| 730 | |||
| 731 | Run everything before pushing: | ||
| 732 | |||
| 733 | ```sh | ||
| 734 | ./scripts/ci.sh | ||
| 735 | ``` | ||
| 80 | Vole is also usable as a Go library. See | ||
| 81 | [Architecture § Using Vole as a library](docs/architecture.md#using-vole-as-a-library). | ||