Sign in

Search issue titles and bodies, change requests and comments. Private repositories you cannot read are not included.

vole
Clone

Read only, no credentials:

https://rodent.software/vole.git

Write over SSH with a registered key:

git clone ssh://git@rodent.software:2222/vole.git

A free and open-source forge that lives in your Git repo.

#60

Highlight with Prism in a worker, and gzip text responses

merged · by keenan 12d ago · merged into main at 306f36fc by keenan 12d ago

keenan opened this 12d ago

Summary

Three changes that only make sense together: JavaScript becomes a requirement, syntax highlighting moves to Prism in a web worker, and the instance gzips text responses so the larger grammar bundle is cheap to ship.

Base is quotas-and-abuse-controls, not main: this work sits on the nine unmerged commits of that branch, and server/web/auth.go is one of the files both touch. Retarget when the lower branch lands:

./vole change update <id> --base main

1. JavaScript is required

The no-script positions are gone rather than maintained:

  • all three <noscript> fallbacks removed — the Save labels and Save assignees buttons (issues.html) and the branch-picker Go button (code.html);
  • the dual-submission path for labels and assignees removed: writes.go reads only the hidden label/assignee fields the chip UI writes, not the comma-separated text field that existed for a browser without JavaScript;
  • tokenPickerData.Field and its four template call sites dropped with it;
  • docs/ui-plan.md states the policy (replacing “Progressive enhancement”), and the comments in highlight.go, tokens.go, app.js, vole.css and ui.go no longer describe a no-JS mode.

Kept deliberately: the htmxRequest fragment/redirect split (plain forms such as issue_new.html post normally and need the redirect), and pager links that navigate the whole page. Neither is a no-script shim.

2. Chroma out, Prism in a worker

  • github.com/alecthomas/chroma/v2 and its regexp2 dependency are gone from go.mod/go.sum. Chroma’s 279 lexers were parsed at process start, which cost every vole CLI invocation ~6 ms; that is gone too.
  • server/web/highlight.go is now an extension/name table — 186 languages, 351 entries — with names matched before extensions (so Dockerfile and nginx.conf win over ini) and the collisions resolved on purpose (.v is Verilog, .m is Objective-C, .h is C, .pp is Puppet, .cls is LaTeX).
  • The blob is served as escaped lines with their numbers and id="L<n>" anchors, exactly as before, plus data-highlight="<language>". One consequence worth noting: highlighted files previously went through Chroma’s own table and had no line anchors at all; now every blob has them.
  • static/prism-worker.js highlights off the main thread. It tokenises the whole blob, then splits the token HTML back into lines by closing open spans at each newline and reopening them on the next, so a block comment or raw string keeps its class without swallowing the following rows. It answers lines: null for an unknown grammar or a grammar that throws, and the page keeps the server’s text. The worker is created only on a page that has a blob.
  • static/app.js joins the td.code contents, posts to the worker, and writes one HTML string per cell; htmx:afterSwap re-runs it when the code browser swaps a file in. Only cell contents change, so there is no layout shift and anchors never move.
  • The bundle is every language Prism 1.30 ships — 293 beyond the four in the core build — concatenated in the dependency order from Prism’s components.json require graph. scripts/vendor-prism.py regenerates it and --check fails if the committed assets drift; a browser or worker that cannot load Prism simply leaves the blob as plain text.
  • The server does not pick a grammar the bundle lacks: TestPrismLanguagesAreVendored holds all 351 table entries to the bundle, so a typo fails the build instead of quietly colouring nothing.
  • CSP gains worker-src 'self' only. No 'wasm-unsafe-eval', no inline styles, no script-src relaxation: Prism emits classes and the palette stays in vole.css. Prism is MIT (prism.LICENSE, NOTICE).

3. Compression and caching

  • server/web/compress.go gzips text-like responses inside the existing middleware: pages, fragments, the stylesheet, scripts, source blobs, and the styled error pages. Not compressed: Accept-Encoding without gzip or with q=0, HEAD, Range requests, 1xx/204/206/304/redirects, already-encoded bodies, non-text types (archives, LFS objects, images, git protocol output), bodies a handler already sized below 256 bytes, and anything a handler marks Cache-Control: no-transform.
  • Flush and Unwrap are preserved, so the streaming handlers and http.ResponseController behave as before.
  • The raw-download route sets no-transform, because it promises a Content-Length the client can trust before the first byte.
  • web.Static() no longer uses http.FileServerFS: an embedded file has no mtime, so assets were served with no validator at all and the 545 KB bundle would be re-fetched on every code page. Each asset now carries a strong ETag from its bytes plus Cache-Control: no-cache, and a miss is a 404 with no directory listing.
  • No new dependency: compress/gzip from the standard library.

Measurements

before after
release binary (-s -w -trimpath) 26.08 MB 22.70 MB
/r/{repo}/code over the wire 11.1 KB 2.6 KB
/r/{repo}/blob/{ref}/{file} 11.1 KB 2.7 KB
styled 404 2.4 KB 0.9 KB
static/prism-languages.min.js — 545 KB raw, 206 KB gzipped, fetched once

Verification

  • gofmt -l, go vet ./..., go test ./... — clean.
  • ./vole workflow run verify — ok: 1 job(s) (format, vet, build, packaging, markdown links, race suite).
  • New tests: 22 compression negotiation cases plus flush and unwrap; compressed page, identity client, static 304/revalidation, static 404 surface, and the two Prism table tests. Four assertions that expected class="chroma" were updated to the new contract (data-highlight, no server-side token spans, source still escaped).
  • The grammar bundle was checked out-of-band against the real files: all 403 registered names load, and every one highlights a mixed sample with no exception, no line-count drift, balanced spans per line, and an exact round-trip. A real rendered blob page was fed through the worker end to end.

Follow-ups, not in this change

  • server/pages (user sites) sits outside Web.Middleware, so sites are still uncompressed; they are static files and would benefit most.
  • Brotli would take the bundle from 206 KB to roughly 150 KB, at the cost of a dependency.
  • An uncommon extension now shows plain text instead of Chroma’s roughly-coloured text; adding one is a line in highlight.go (the test proves the grammar is there).
  • The author cannot approve this request; it needs a second account.

Discussion (0)

No discussion yet.