Sign in

Search issue titles and bodies, change requests and comments. Private repositories you cannot read are not included.

vole
Clone

Read only, no credentials:

https://rodent.software/git/vole.git

Write over SSH with a registered key:

git clone ssh://git@0.0.0.0:2222/vole.git

A free and open-source forge that lives in your Git repo.

Apply a push's index delta as the push lands, and check the ref by reading it

dced06da1bfc4a8e542e5d86d67888362077ed39 · Keenan Nemetz · 9d ago · parent 26c9be33

Apply a push's index delta as the push lands, and check the ref by reading it

A push's metadata delta used to be applied by the next reader, so the
first page after a big push paid for the whole push. And search checked
every visible repository before querying: a `symbolic-ref` to open it and
a `rev-parse` to see whether the metadata branch moved, so N repositories
cost 2N git processes per search.

- Post-receive now opens the index (the SSH server hands it `VOLE_INDEX_DB`
  and the repository id) and runs Ensure after the refs have moved, which
  is also after a followed request's head has moved. A failure is reported
  and ignored: the refs have landed, and the next read still repairs it.
- `git.Repo.RefCommit` resolves a branch ref by reading the loose ref file
  git writes for every branch update, and falls back to `rev-parse` for a
  packed ref, a symbolic ref, or a missing one. The index's Ensure and the
  web's metadata-head read use it.
- `repo.Store.Meta` builds a repository handle without a HEAD lookup, and
  search uses it: the refresh needs the directory and the ACL, not the
  default branch. The per-repository check is now two file reads instead
  of two processes.

Load harness, `search-find` at concurrency 4 over three repositories:
510.2 req/s and p50 7 ms before, 814.8 req/s and p50 4 ms after. A push
that changes the `vole` branch leaves the rows current before the push
returns; a test queries them with no Ensure in between.

server/hook/hook.go +26

@@ -17,6 +17,7 @@ import (
17 17 "github.com/rodent-software/vole/core/lfs"
18 18 "github.com/rodent-software/vole/forge/access"
19 19 "github.com/rodent-software/vole/forge/identity"
20 "github.com/rodent-software/vole/forge/index"
20 21 "github.com/rodent-software/vole/forge/landing"
21 22 "github.com/rodent-software/vole/forge/repo"
22 23 "github.com/rodent-software/vole/internal/audit"
@@ -52,6 +53,10 @@ type Config struct {
52 53 // RepoID is the repository's stable id, which is how the LFS store keys its
53 54 // links. Empty disables the LFS quota.
54 55 RepoID string
56 // IndexDB, with RepoID, lets post-receive bring the read model up to the
57 // branch the push left behind, so the next page does not apply the delta.
58 // Empty disables the refresh.
59 IndexDB string
55 60 // AuditPath receives lines about privileged metadata operations. When it is
56 61 // empty or unwritable, an override is refused and an ordinary push is answered
57 62 // with a warning: the refs have already moved by the time post-receive runs.
@@ -709,9 +714,30 @@ func PostReceive(cfg Config) int {
709 714 fmt.Fprintf(cfg.Out, "vole: could not queue CI for %s: %v\n", ref, err)
710 715 }
711 716 }
717 if err := RefreshIndex(cfg); err != nil {
718 fmt.Fprintf(cfg.Out, "vole: could not refresh the index: %v\n", err)
719 }
712 720 return 0
713 721 }
714 722
723 // RefreshIndex brings the derived rows up to the metadata branch the push left
724 // behind. It runs in post-receive, after the ref update and after a followed
725 // request's head has moved, so a page or a search finds the work already applied.
726 //
727 // A failure is reported and ignored: the refs have moved and the push cannot be
728 // failed now, and the next read still checks the branch and repairs the rows.
729 func RefreshIndex(cfg Config) error {
730 if cfg.IndexDB == "" || cfg.RepoID == "" {
731 return nil
732 }
733 store, err := index.Open(cfg.IndexDB)
734 if err != nil {
735 return err
736 }
737 defer store.Close()
738 return store.Ensure(&repo.Meta{ID: cfg.RepoID, Dir: cfg.RepoDir})
739 }
740
715 741 // shortOrNone renders a sha for the log, naming the two states that are not a commit:
716 742 // a ref being created and one being deleted.
717 743 func shortOrNone(sha string) string {