vole
Clone
Read only, no credentials:
https://rodent.software/git/vole.git
Write over SSH with a registered key:
git clone ssh://git@0.0.0.0:2222/vole.git
A free and open-source forge that lives in your Git repo.
Store CI run records in the database, not the repository
b312e1660ac3d570fc35350a22b3514c56ab466f · Keenan Nemetz · 11d ago · parent db5e4382
Store CI run records in the database, not the repository A finished attempt was a set of files on the vole branch: a record, step ranges, and artifact pointers, pushed as a commit. At 959 files, CI records were 849 of them, and every human metadata write had to carry them. The runner now sends the record with its finish message over the SSH channel that holds the lease. The server checks the account, the job, and the log and artifact pointers, then writes the attempt, steps, artifacts, attestation, and run result in one transaction in data/runs.db. A tagged build still pushes its release pointers to the branch after the record is stored. Pre-receive refuses a new runs/ file. vole admin index rebuild imports records still on an old branch; records written since are never deleted by a rebuild. The run page reads the stored attempt, and runs.db is now a source that must be backed up. This stops new CI history from entering Git. Existing commits keep the records they already have; removing those would require rewriting the protected vole branch, which this change does not do.
docs/runner.md +20−17
| @@ -50,7 +50,7 @@ vole runner --remote ssh://git@host:2222 --known-hosts ~/.ssh/vole_known_hosts | |||
| 50 | 50 | including `vole attach`, `vole admin --remote`, and the CLI verbs. | |
| 51 | 51 | ||
| 52 | 52 | The policy covers git's own ssh calls as well as the runner's control channel, so | |
| 53 | a clone, a fetch, a record push and the job stream all verify the same host. An | ||
| 53 | a clone, a fetch, a release push and the job stream all verify the same host. An | ||
| 54 | 54 | environment variable cannot loosen it: an explicit | |
| 55 | 55 | `GIT_SSH_COMMAND=... StrictHostKeyChecking=no` is overridden by the runner's own | |
| 56 | 56 | options, which come first. | |
| @@ -125,7 +125,7 @@ that build. Prefer rootless Podman, and pin the base image by digest. | |||
| 125 | 125 | ||
| 126 | 126 | One container is created per job and removed afterward. The workflow's steps run | |
| 127 | 127 | inside it, but the runner process stays on the host: the SSH key, the trusted Git | |
| 128 | cache, and the record push are never exposed to the job. Each job gets a fresh | ||
| 128 | cache, and the finish channel are never exposed to the job. Each job gets a fresh | ||
| 129 | 129 | clone, so a job cannot leave a Git hook or a modified checkout behind for the next | |
| 130 | 130 | one. | |
| 131 | 131 | ||
| @@ -246,14 +246,16 @@ Admins always count. For everyone else, a maintainer running CI on their laptop | |||
| 246 | 246 | not turn a required check green. The result is still recorded, and the run page | |
| 247 | 247 | says whether it counts. | |
| 248 | 248 | ||
| 249 | When a job finishes, the runner uploads its log, writes a record under | ||
| 250 | `runs/<run id>/<attempt>/`, and pushes the record together with artifact pointers | ||
| 251 | and any release pointers. Vole checks that the account is allowed, that the record | ||
| 252 | names that account, and that it describes the job the account was given. The run | ||
| 253 | page names the account and links to the commit that holds the record. | ||
| 249 | When a job finishes, the runner uploads its log and sends the attempt's record with | ||
| 250 | its finish message on the same SSH channel that leased the job. Vole checks that the | ||
| 251 | record names the account holding the lease, that it describes the job the queue | ||
| 252 | handed out, and that every log and artifact pointer exists in the store. It then | ||
| 253 | stores the attempt and the run's result in `data/runs.db` in one transaction. A | ||
| 254 | tagged build that publishes release artifacts also pushes those pointers to the | ||
| 255 | `vole` branch after the record is stored. | ||
| 254 | 256 | ||
| 255 | The runner did the work, so the runner reports the result. The SSH key behind the | ||
| 256 | push identifies the runner, and `data/audit.log` records the decision. | ||
| 257 | The runner did the work, so the runner reports the result. The SSH connection | ||
| 258 | identifies the runner, and `data/audit.log` records the decision. | ||
| 257 | 259 | ||
| 258 | 260 | ## Logs | |
| 259 | 261 | ||
| @@ -263,9 +265,9 @@ runner's own record. Once the attempt finishes, the archived object is served | |||
| 263 | 265 | directly and supports range requests, so the page does not read a large log into | |
| 264 | 266 | memory. | |
| 265 | 267 | ||
| 266 | The record also names each step's byte range in that one log, so the run page can | ||
| 267 | show a step's output without downloading the image build and other steps. Step | ||
| 268 | timing is recorded beside the range. | ||
| 268 | The stored attempt also names each step's byte range in that one log, so the run | ||
| 269 | page can show a step's output without downloading the image build and other steps. | ||
| 270 | Step timing is recorded beside the range. | ||
| 269 | 271 | ||
| 270 | 272 | ```text | |
| 271 | 273 | /r/<repo>/runs/<id>/log # latest attempt | |
| @@ -288,8 +290,8 @@ URL redirects there. | |||
| 288 | 290 | ||
| 289 | 291 | How the queue behaves: | |
| 290 | 292 | ||
| 291 | - Finished runs are stored as records on the `vole` branch. `data/runs.db` only | ||
| 292 | stores work that has not happened yet. | ||
| 293 | - Finished runs are stored in `data/runs.db`, not on the `vole` branch. A clone or | ||
| 294 | backup of the repository does not carry CI history, so back up `runs.db`. | ||
| 293 | 295 | - A push normally reaches the queue in under a second. Every 30 seconds Vole also | |
| 294 | 296 | compares each branch and tag with the last tip it queued, so a missed | |
| 295 | 297 | notification still produces a run. | |
| @@ -297,6 +299,7 @@ How the queue behaves: | |||
| 297 | 299 | run that is already in progress. | |
| 298 | 300 | - If a runner disappears, its job is requeued. After five attempts the job fails | |
| 299 | 301 | instead of cycling forever. | |
| 300 | - Deleting `data/runs.db` loses attempts that were in flight. Current branch and | ||
| 301 | tag tips are queued again from Git, and `vole admin index rebuild` restores | ||
| 302 | finished history from the records on each repository's `vole` branch. | ||
| 302 | - Deleting `data/runs.db` loses CI history and attempts that were in flight. Current | ||
| 303 | branch and tag tips are queued again from Git, but only a backup can restore the | ||
| 304 | finished runs. `vole admin index rebuild` imports records still on an older | ||
| 305 | repository's `vole` branch; it is the upgrade path, not a backup. | ||