Why
Stage 2 of #113. Stage 1 (#114) indexed the branch’s lists and counts; the text
is now in index.db too, so one query can find an issue title, a change title
and a comment body without reading a repository’s metadata branch. Without
ranking over comment bodies, “search everything” is a feature that gets rewritten
later, so this uses FTS5 now.
Stacked on #114 (--base index/metadata-branch), which is stacked on #112.
Retarget to main after those land.
What changed
searchis an FTS5 table fed by the same refresh as the list rows: one document per issue, change request and comment (title, body, author). A comment’s row carries its parent’s title and id, so a result can say what it is on and link to the fragment in one read.- An entity’s documents are replaced with the entity and its comments, and a deletion removes both — an edited or deleted comment cannot survive in a query.
[#42](/r/vole/n/42)and an id prefix are answered fromentity_numbersbefore the text search, so the thing someone named is the first row. The text query quotes every token and prefix-matches only the last, so punctuation in the search box cannot change the query’s meaning.GET /searchis one page for the instance, linked from the header next to Activity. Results show kind, repository, number, author, and the matched sentence with the match marked (escaped before the markers become tags).- Visibility is the page rule. The handler passes the query only the repository ids the request may read; a private repository’s titles never enter the result set. A test proves it by indexing a private issue as its owner and then searching anonymously.
- Freshness does not depend on a page having been visited. A search refreshes the repositories it may see first, so an issue pushed a minute ago is found even if nobody opened that repository since.
Build
FTS5 needs -tags sqlite_fts5. The verify and release images, both workflows,
README.md, docs/getting-started.md and CONTRIBUTING.md now pass it beside
libsqlite3; with the system library the tag is inert and the module is whatever
the distribution built (Debian, Ubuntu, Fedora, RHEL and Rocky all enable FTS5,
and docs/packaging.md now says the dependency includes it).
Local commands:
go build -tags libsqlite3,sqlite_fts5 -o vole ./cmd/vole
go test -tags libsqlite3,sqlite_fts5 -race ./...
The libsqlite3-tagged build cannot run on this machine (no sqlite3.h), so the
checks below used the bundled SQLite:
GOFLAGS=-tags=sqlite_fts5 CGO_ENABLED=1 go test -race ./...
Checks
./vole workflow check,gofmt,go vet ./..., full-racesuite with the tag.- New tests: search finds all three kinds and links comments to their thread,
scope is exactly the ids given,
[#42](/r/vole/n/42)and id prefixes sort first, deleted comments and entities disappear,IndexReporebuilds the same documents, and an anonymous search cannot see a private repository’s issue.
Not in this change
- Stage 3: detail pages and the activity feed still read git; #99’s harness should measure before deciding whether they move.
- A per-repository unified search page. The existing per-repo filters (issues, changes, code search) are unchanged; a repo-scoped universal result can be a follow-up if it is wanted.
- A search refresh still costs one
rev-parseper visible repository. A push-time refresh from the pre-receive hook would remove that, and is its own change.