Sign in

Search issue titles and bodies, change requests and comments. Private repositories you cannot read are not included.
vole
Clone

Read only, no credentials:

https://rodent.software/vole.git

Write over SSH with a registered key:

git clone ssh://git@rodent.software:2222/vole.git

A free and open-source forge that lives in your Git repo.

#91

P2: Shrink the binary by linking SQLite dynamically through cgo

closed · opened by keenan 10d ago

keenan opened this 10d ago

modernc.org/sqlite is a pure-Go translation of SQLite, and it is the largest thing in the binary. Measured with the release flags (-trimpath -ldflags "-s -w -buildid=", linux/amd64, go1.27.3):

Build Size vs today
today: CGO_ENABLED=0 + modernc 23,085,180 B (22.0 MiB) —
cgo + mattn/go-sqlite3, SQLite compiled in 20,570,752 B (19.6 MiB) −2.4 MiB (−10.9%)
cgo + mattn/go-sqlite3 -tags libsqlite3 (system libsqlite3.so.0) 19,345,008 B (18.4 MiB) −3.6 MiB (−16.2%)

SQLite alone is ~4.03 MB of today’s binary (a build with the driver stubbed out is 19.05 MB); the cgo binding against the system library costs ~0.25 MB. The cgo driver is also typically several times faster than the modernc translation on database-heavy paths, which matters for forge/index on a large repository.

Why now: the release tarball and the .deb/.rpm are the main distribution channel, and a 16% smaller download with faster queries is a straight win. The cost is that the binary is no longer self-contained: it needs libsqlite3.so.0 at runtime, which the packages must declare.

Scope

  • Replace modernc.org/sqlite with github.com/mattn/go-sqlite3 in forge/index, forge/access, forge/identity, forge/session, forge/pageview, core/lfs, and ci/runs.
  • Rewrite the six DSNs: modernc-only _pragma=busy_timeout(5000) / _pragma=journal_mode(WAL) / _pragma=foreign_keys(1) become _busy_timeout=5000 / _journal_mode=WAL / _foreign_keys=1; the driver name becomes sqlite3.
  • CGO_ENABLED=1 -tags libsqlite3 everywhere: the verify image, the verify workflow steps, the release workflow, and the build instructions.
  • Cross-compile arm64 in the release image with gcc-aarch64-linux-gnu and libsqlite3-dev:arm64; keep both architectures in one job.
  • Declare the runtime dependency in packaging/nfpm.yaml: libsqlite3-0 for deb, sqlite-libs for rpm.
  • Document the new build prerequisites (C compiler, libsqlite3-dev / sqlite-devel) and the runtime library.

Acceptance criteria

  • vole workflow run verify passes, with the test suite running against the system SQLite rather than the amalgamation.
  • The release build produces both architectures with libsqlite3.so.0 in their NEEDED entries, and libsqlite3-0 / sqlite-libs in the package metadata.
  • dist/vole-linux-amd64 is materially smaller than the 23.1 MB the current release flags produce.
  • go build -tags libsqlite3 ./cmd/vole and the documented instructions agree.

Comments (0)

No comments yet.