Sign in

Search issue titles and bodies, change requests and comments. Private repositories you cannot read are not included.

vole
Clone

Read only, no credentials:

https://rodent.software/git/vole.git

Write over SSH with a registered key:

git clone ssh://git@0.0.0.0:2222/vole.git

A free and open-source forge that lives in your Git repo.

Phase 4: the runner uploads the log, so the server cannot write it

86fe5bb49ddeeb89a5d58459f27a86a83f82481a · Keenan Nemetz · 17d ago · parent 9cc2f087

Phase 4: the runner uploads the log, so the server cannot write it

Streaming made the server the author of bytes the runner produced. Heartbeats
carried output chunks; the server concatenated them into a SQLite column and, at
finish, hashed the result itself with `LFS.PutBytes`. So the archived object's id was
the server's, not the writer's — the one output in the system that worked that way,
since artifacts are uploaded by the runner through put-object and pages/releases are
pushed by it. Nothing depended on it, which is why it was harmless; a signed run
record could never have bound it, which is why it had to go.

Streaming is dropped. At finish the runner uploads its log through the same
git-lfs-transfer path artifacts use, gets the object id the store computed from the
content, and names it in the finish message. The server's rule is the artifact rule:
the store is the authority on whether the bytes exist, and a claimed size that is not
the object's size is refused. Heartbeats now carry nothing but the lease.

A log is per attempt, not per run. A job that kills a runner is requeued, so one run
can have several logs, and the interesting one is usually the attempt that died:
`run_logs(run_id, attempt, oid, size)` replaces the single column, `AttemptLogs`
reads them oldest first, `/runs/<id>/log` serves the newest and `?attempt=N` picks
one, and `vole gc` treats every attempt's object as a root. Rows written before this
change still have their `log_oid`, which reads keep honouring; `Store.Log` and
`SetLogOID` are gone with the code that used them.

The cost, stated plainly: there is no live tail any more, and an attempt whose runner
was killed leaves no log at all — the attempt that hung is exactly the log somebody
wants. That is the trade: a log that can be shown to be the runner's is worth more
than one that arrives early. It also removes an unbounded text column from the queue,
which was growing into the WAL and every backup until a run finished.

Verified: TestCheckRunLogTrustsTheStore pins the new rule at the boundary (stored log
accepted, size mismatch refused, object absent refused, non-oid refused, no store
refused); TestLogsAreRecordedPerAttempt shows two attempts recorded separately,
`Get` returning the newest, both objects named for the sweep, and a finished run
refusing a log; TestHeartbeatAndFinishNeedTheLease now proves a stranger can neither
heartbeat, nor finish, nor record a log, and that a finished run carries no log text;
and the end-to-end test that runs the real binary now asserts the served log hashes
to the object id in the run row and that the queue kept no copy of it. scripts/ci.sh
green (gofmt, vet, build, markdown links, go test -race ./...).

README.md +11−5

@@ -47,7 +47,7 @@ Working today:
47 47 page with its patch, and a change request with the diff that would land
48 48 - CI: workflows in `.vole/workflows/*.yaml`, validated by the push that introduces
49 49 them. A push queues jobs; `vole runner` leases one over SSH with the same key, runs
50 the shell steps, heartbeats and reports the log. A job can declare `artifacts:`
50 the shell steps and uploads the attempt's log. A job can declare `artifacts:`
51 51 (path patterns), which the runner uploads and the run page serves. Protected
52 52 branches can require green checks as well as approvals
53 53 - LFS: large files stay out of the git object database and live in one
@@ -358,10 +358,16 @@ The metadata commit names the build (`release: v1.0.0 artifacts from run 01M5...
358 358 publish — `git tag -a`, not `git tag` — because a tag with no message has no release
359 359 notes and no tagger. A run for a tag never counts as a branch's required check, even
360 360 when the tag points at that branch's tip. The runner leases a job over the same SSH key you push with,
361 checks out the commit, runs the steps with per-step timeouts, heartbeats while it
362 works, and reports the exit code and log. A runner that disappears has its job
363 requeued; after five attempts the job fails instead of cycling forever. Protected
364 branches can require green checks as well as approvals.
361 checks out the commit, runs the steps with per-step timeouts, and uploads the
362 attempt's log to the object store before reporting the exit code and the log's
363 object id. Nothing is streamed: a log the server assembled would be a log the
364 server wrote, and the runner is the party that can be held to it. `/r/<repo>/runs/<id>/log`
365 serves it, `?attempt=N` picks one attempt out of a retried job, and an attempt whose
366 runner was killed has no log at all.
367
368 A runner that disappears has its job requeued; after five attempts the job fails
369 instead of cycling forever. Protected branches can require green checks as well as
370 approvals.
365 371
366 372 `artifacts:` declares the files a run publishes. The patterns are expanded in the
367 373 checkout after the steps pass, uploaded over SSH into the same content-addressed