Sign in

Search issue titles and bodies, change requests and comments. Private repositories you cannot read are not included.
vole
Clone

Read only, no credentials:

https://rodent.software/vole.git

Write over SSH with a registered key:

git clone ssh://git@rodent.software:2222/vole.git

A free and open-source forge that lives in your Git repo.

Move writes to SSH, drop web authentication, keep identity in git

a5f19b3c19861e080953096f934df25dc38beaf7 · Keenan Nemetz · 18d ago · parent 653e0cfc 

Move writes to SSH, drop web authentication, keep identity in git

The authenticated-HTTP design is gone rather than extended. Writes now require
an SSH key, the web is read-only, and the user roster lives in git.

What replaced what:

- internal/sshserver is the only write transport. An embedded SSH server
  (golang.org/x/crypto/ssh) authenticates by public key and hands the principal
  to pre-receive through the environment, so the hook's policy code is unchanged.
  The exec surface is whitelisted to git-upload-pack and git-receive-pack, the
  repository argument is resolved through the store rather than concatenated, and
  everything else -- including `sh -c` -- is refused and asserted refused in a
  test.
- internal/identity stores users on refs/heads/main in data/system.git. The
  directory listing is the table: users/<name>/keys/<sha>.pub. Writes are CAS
  commits, reads come from an in-memory snapshot so the SSH auth callback is a
  map lookup. Nothing stored is a secret, so a clone of system.git is a complete
  backup and git log on it is the audit trail.
- internal/transport is now anonymous read-only: upload-pack behind a visibility
  check, 404 for private repositories, and no push path at all.
- internal/web has no login, no sessions and no mutations.

Deleted outright: internal/auth, the Basic-auth path, login/logout, web
settings/rename/delete, chunked body spooling, the spool janitor, the push
semaphore and the hand-rolled byte cap. The cap is now git's receive.maxInputSize
per repo, which enforces it on every transport without buffering.

Three things worth recording:

- The CAS helper read the tree outside the retry loop and compared against a
  freshly-read old value, so a concurrent writer's commit could be silently
  discarded while the CAS "succeeded". A test with two stores adding eight keys
  at once found it losing half of them; read, mutate and compare now all happen
  inside the loop.
- The roster moved from refs/vole/users to refs/heads/main because `git clone`
  only mirrors refs/heads/*: the documented backup command produced an instance
  with zero users. Storing on a branch also means git itself refuses to point the
  ref at a non-commit.
- Exec refusals are answered on the channel rather than by rejecting the request.
  Rejecting made the client print "exec request failed on channel 0", so the
  reason was unreadable.

Verified by hand as well as by test: init -> user -> add-key -> repo -> serve,
push over SSH, anonymous HTTP clone, web browse, a private repo answering 404 on
both anonymous surfaces while still cloning over SSH, a refused `sh -c` that
prints why and executes nothing, and a plain clone of system.git yielding the
whole roster with no side database present.

internal/identity/identity.go +476

@@ -0,0 +1,476 @@
1 // Package identity keeps the instance's user roster in git, inside a dedicated
2 // system repository.
3 //
4 // The directory listing is the table: users/<name>/keys/<hex>.pub. Everything
5 // stored here is public material, so a copy of the system repo is a complete
6 // backup that leaks nothing. Writes are compare-and-swap commits; reads come
7 // from an in-memory snapshot refreshed when the ref moves, because the SSH auth
8 // callback is on the hot path and must be a map lookup, not a git invocation.
9 //
10 // The roster lives on refs/heads/main on purpose. A custom namespace such as
11 // refs/vole/users is invisible to a plain `git clone`, which only mirrors
12 // refs/heads/*, so the obvious backup command would have silently produced an
13 // instance with no users. The system repo is never pushed to by clients, so it
14 // gains nothing from a reserved namespace.
15 package identity
16
17 import (
18 "crypto/rand"
19 "crypto/sha256"
20 "encoding/hex"
21 "encoding/json"
22 "fmt"
23 "os"
24 "path/filepath"
25 "sort"
26 "strings"
27 "sync"
28 "sync/atomic"
29 "time"
30
31 "golang.org/x/crypto/ssh"
32
33 "github.com/rodent-software/vole/internal/gitx"
34 )
35
36 const (
37 UsersRef = "refs/heads/main"
38 usersDir = "users"
39 metaFile = "meta.json"
40 keysDir = "keys"
41 maxCAS = 64
42 DirName = "system.git"
43 )
44
45 type User struct {
46 Name string
47 ID string
48 Created time.Time
49 Disabled bool
50 Keys []Key
51 }
52
53 type Key struct {
54 Fingerprint string // ssh.FingerprintSHA256 of the key
55 Type string
56 Comment string
57 Line string // authorized_keys-format line
58 Added time.Time
59 }
60
61 // Snapshot is an immutable view of the roster, swapped atomically.
62 type Snapshot struct {
63 Users map[string]*User
64 ByFingerprint map[string]string
65 SHA string
66 }
67
68 func (s *Snapshot) Lookup(fingerprint string) (string, bool) {
69 if s == nil {
70 return "", false
71 }
72 name, ok := s.ByFingerprint[fingerprint]
73 return name, ok
74 }
75
76 func (s *Snapshot) Names() []string {
77 if s == nil {
78 return nil
79 }
80 out := make([]string, 0, len(s.Users))
81 for n := range s.Users {
82 out = append(out, n)
83 }
84 sort.Strings(out)
85 return out
86 }
87
88 type Store struct {
89 Dir string
90 Git *gitx.Repo
91 snap atomic.Pointer[Snapshot]
92 mu sync.Mutex // serialises local writers
93 }
94
95 // Init creates the system repo with an empty roster.
96 func Init(dir string) error {
97 if _, err := os.Stat(dir); err == nil {
98 return nil
99 }
100 if err := gitx.InitBare(dir, "main"); err != nil {
101 return err
102 }
103 s, err := Open(dir)
104 if err != nil {
105 return err
106 }
107 return s.writeTree(map[string][]byte{
108 usersDir + "/.keep": []byte("the users tree: one directory per user\n"),
109 }, "users: initialise empty roster", "vole")
110 }
111
112 // Open loads an existing system repo. A missing or unreadable roster is a hard
113 // error: booting with an empty roster would silently mean "nobody can
114 // authenticate", and the diagnosis would happen at an inconvenient hour.
115 func Open(dir string) (*Store, error) {
116 g := gitx.Open(dir)
117 if !g.Exists() {
118 return nil, fmt.Errorf("identity: %s is missing; run vole init", dir)
119 }
120 s := &Store{Dir: dir, Git: g}
121 if err := s.Reload(); err != nil {
122 return nil, err
123 }
124 return s, nil
125 }
126
127 // Reload re-reads the roster from the ref.
128 func (s *Store) Reload() error {
129 sha, err := s.Git.RevParseRaw(UsersRef)
130 if err != nil {
131 return err
132 }
133 snap := &Snapshot{Users: map[string]*User{}, ByFingerprint: map[string]string{}, SHA: sha}
134 if sha == "" {
135 s.snap.Store(snap)
136 return nil
137 }
138 // A ref pointing at anything but a commit means the roster is corrupt. Left
139 // unchecked, `ls-tree` on a blob prints nothing and the instance would boot
140 // with an empty roster — meaning nobody can authenticate.
141 typ, err := s.Git.ObjectType(sha)
142 if err != nil {
143 return fmt.Errorf("identity: read %s: %w", UsersRef, err)
144 }
145 if typ != "commit" {
146 return fmt.Errorf("identity: %s points at a %s, want a commit", UsersRef, typ)
147 }
148 entries, err := s.Git.LsTreeR(sha)
149 if err != nil {
150 return fmt.Errorf("identity: read roster: %w", err)
151 }
152 for _, e := range entries {
153 if e.Type != "blob" || !strings.HasSuffix(e.Name, ".pub") {
154 continue
155 }
156 name := userNameFromPath(e.Name)
157 if name == "" {
158 continue
159 }
160 data, err := s.Git.BlobBySHA(e.SHA)
161 if err != nil {
162 return fmt.Errorf("identity: read key %s: %w", e.Name, err)
163 }
164 pub, comment, _, rest, err := ssh.ParseAuthorizedKey(data)
165 if err != nil {
166 return fmt.Errorf("identity: %s: unparsable key: %w", e.Name, err)
167 }
168 _ = rest
169 u := snap.Users[name]
170 if u == nil {
171 u = &User{Name: name}
172 snap.Users[name] = u
173 }
174 fp := ssh.FingerprintSHA256(pub)
175 u.Keys = append(u.Keys, Key{
176 Fingerprint: fp,
177 Type: pub.Type(),
178 Comment: comment,
179 Line: strings.TrimSpace(string(data)),
180 })
181 snap.ByFingerprint[fp] = name
182 }
183 // Metadata files.
184 for _, e := range entries {
185 if e.Type != "blob" || !strings.HasSuffix(e.Name, "/"+metaFile) {
186 continue
187 }
188 name := userNameFromPath(e.Name)
189 if name == "" {
190 continue
191 }
192 data, err := s.Git.BlobBySHA(e.SHA)
193 if err != nil {
194 return err
195 }
196 var m struct {
197 ID string `json:"id"`
198 Created time.Time `json:"created"`
199 Disabled bool `json:"disabled"`
200 }
201 if err := json.Unmarshal(data, &m); err != nil {
202 return fmt.Errorf("identity: %s/%s: %w", name, metaFile, err)
203 }
204 u := snap.Users[name]
205 if u == nil {
206 u = &User{Name: name}
207 snap.Users[name] = u
208 }
209 u.ID, u.Created, u.Disabled = m.ID, m.Created, m.Disabled
210 }
211 // Order keys for stable output.
212 for _, u := range snap.Users {
213 sort.Slice(u.Keys, func(i, j int) bool { return u.Keys[i].Fingerprint < u.Keys[j].Fingerprint })
214 }
215 s.snap.Store(snap)
216 return nil
217 }
218
219 // Snapshot returns the current roster view. Cheap and lock-free.
220 func (s *Store) Snapshot() *Snapshot { return s.snap.Load() }
221
222 // Lookup resolves a key fingerprint to a username.
223 func (s *Store) Lookup(fingerprint string) (string, bool) {
224 name, ok := s.Snapshot().Lookup(fingerprint)
225 if !ok {
226 return "", false
227 }
228 if u, ok := s.Snapshot().Users[name]; ok && u.Disabled {
229 return "", false
230 }
231 return name, true
232 }
233
234 func ValidateName(name string) error {
235 if len(name) < 2 || len(name) > 32 {
236 return fmt.Errorf("username must be 2-32 characters")
237 }
238 for _, r := range name {
239 if (r >= 'a' && r <= 'z') || (r >= '0' && r <= '9') || r == '-' || r == '_' {
240 continue
241 }
242 return fmt.Errorf("username may only contain lowercase letters, digits, dash and underscore")
243 }
244 return nil
245 }
246
247 // Create adds a user with no keys. They cannot authenticate until a key is
248 // added, which is the intended order of operations.
249 func (s *Store) Create(name string) error {
250 if err := ValidateName(name); err != nil {
251 return err
252 }
253 if _, ok := s.Snapshot().Users[name]; ok {
254 return fmt.Errorf("identity: user %q already exists", name)
255 }
256 id, err := randomID()
257 if err != nil {
258 return err
259 }
260 meta, err := json.MarshalIndent(struct {
261 ID string `json:"id"`
262 Created time.Time `json:"created"`
263 Disabled bool `json:"disabled"`
264 }{id, time.Now().UTC(), false}, "", " ")
265 if err != nil {
266 return err
267 }
268 return s.apply(func(files map[string][]byte) error {
269 files[fmt.Sprintf("%s/%s/%s", usersDir, name, metaFile)] = append(meta, '\n')
270 return nil
271 }, fmt.Sprintf("users: create %s", name), "vole")
272 }
273
274 // AddKey registers an SSH public key for a user and returns its fingerprint.
275 func (s *Store) AddKey(name, authorizedKey, addedBy string) (string, error) {
276 pub, _, _, _, err := ssh.ParseAuthorizedKey([]byte(strings.TrimSpace(authorizedKey)))
277 if err != nil {
278 return "", fmt.Errorf("identity: not an SSH public key: %w", err)
279 }
280 fp := ssh.FingerprintSHA256(pub)
281 if owner, ok := s.Lookup(fp); ok {
282 return "", fmt.Errorf("identity: that key is already registered to %s", owner)
283 }
284 if _, ok := s.Snapshot().Users[name]; !ok {
285 return "", fmt.Errorf("identity: no such user %q", name)
286 }
287 line := strings.TrimSpace(authorizedKey) + "\n"
288 path := fmt.Sprintf("%s/%s/%s/%s.pub", usersDir, name, keysDir, keyFileName(pub))
289 err = s.apply(func(files map[string][]byte) error {
290 files[path] = []byte(line)
291 return nil
292 }, fmt.Sprintf("users: add key for %s (%s)", name, fp), addedBy)
293 if err != nil {
294 return "", err
295 }
296 return fp, nil
297 }
298
299 // RemoveKey revokes a key. Recovery is an operator adding a new key, which is
300 // why there is no reset flow to build.
301 func (s *Store) RemoveKey(name, fingerprint, removedBy string) error {
302 u, ok := s.Snapshot().Users[name]
303 if !ok {
304 return fmt.Errorf("identity: no such user %q", name)
305 }
306 var target *Key
307 for i := range u.Keys {
308 if u.Keys[i].Fingerprint == fingerprint {
309 target = &u.Keys[i]
310 break
311 }
312 }
313 if target == nil {
314 return fmt.Errorf("identity: %s has no key %s", name, fingerprint)
315 }
316 pub, _, _, _, err := ssh.ParseAuthorizedKey([]byte(target.Line))
317 if err != nil {
318 return err
319 }
320 path := fmt.Sprintf("%s/%s/%s/%s.pub", usersDir, name, keysDir, keyFileName(pub))
321 return s.apply(func(files map[string][]byte) error {
322 if _, ok := files[path]; !ok {
323 return fmt.Errorf("identity: key file vanished: %s", path)
324 }
325 delete(files, path)
326 return nil
327 }, fmt.Sprintf("users: revoke key for %s (%s)", name, fingerprint), removedBy)
328 }
329
330 // apply is the single write path: read the tree, mutate a flat path map, write
331 // it back with a compare-and-swap ref update, then refresh the snapshot.
332 func (s *Store) apply(mutate func(map[string][]byte) error, message, actor string) error {
333 s.mu.Lock()
334 defer s.mu.Unlock()
335
336 // The read, the mutation and the compare must all see the same ref value.
337 // Reading the tree once outside this loop and then CAS-ing against a fresh
338 // old value silently discards a concurrent writer's commit, because the CAS
339 // succeeds while the tree content is stale.
340 var lastErr error
341 for attempt := 0; attempt < maxCAS; attempt++ {
342 old, err := s.Git.RevParseRaw(UsersRef)
343 if err != nil {
344 return err
345 }
346 files, err := s.readFlat(old)
347 if err != nil {
348 return err
349 }
350 if err := mutate(files); err != nil {
351 return err
352 }
353 tree, err := buildTree(s.Git, files)
354 if err != nil {
355 return err
356 }
357 commit, err := s.Git.CommitTree(tree, old, withActor(message, actor))
358 if err != nil {
359 return err
360 }
361 ok, err := s.Git.UpdateRefCAS(UsersRef, commit, old)
362 if err != nil {
363 return err
364 }
365 if ok {
366 return s.Reload()
367 }
368 lastErr = fmt.Errorf("identity: lost %d ref race(s)", attempt+1)
369 time.Sleep(time.Duration(attempt+1) * 5 * time.Millisecond)
370 }
371 return lastErr
372 }
373
374 func (s *Store) readFlat(sha string) (map[string][]byte, error) {
375 out := map[string][]byte{}
376 if sha == "" {
377 return out, nil
378 }
379 if typ, err := s.Git.ObjectType(sha); err != nil {
380 return nil, err
381 } else if typ != "commit" {
382 return nil, fmt.Errorf("identity: %s points at a %s, want a commit", UsersRef, typ)
383 }
384 entries, err := s.Git.LsTreeR(sha)
385 if err != nil {
386 return nil, err
387 }
388 for _, e := range entries {
389 if e.Type != "blob" {
390 continue
391 }
392 data, err := s.Git.BlobBySHA(e.SHA)
393 if err != nil {
394 return nil, err
395 }
396 out[e.Name] = data
397 }
398 return out, nil
399 }
400
401 // writeTree is the one-shot path used by Init, where the ref cannot exist yet.
402 func (s *Store) writeTree(files map[string][]byte, message, actor string) error {
403 tree, err := buildTree(s.Git, files)
404 if err != nil {
405 return err
406 }
407 commit, err := s.Git.CommitTree(tree, "", withActor(message, actor))
408 if err != nil {
409 return err
410 }
411 _, err = s.Git.UpdateRefCAS(UsersRef, commit, "")
412 return err
413 }
414
415 func withActor(message, actor string) string {
416 if strings.TrimSpace(actor) == "" {
417 actor = "unknown"
418 }
419 return message + "\n\nBy: " + actor
420 }
421
422 // buildTree writes a nested tree from flat paths.
423 func buildTree(g *gitx.Repo, files map[string][]byte) (string, error) {
424 var entries []gitx.Entry
425 dirs := map[string]map[string][]byte{}
426 for path, content := range files {
427 parts := strings.SplitN(path, "/", 2)
428 if len(parts) == 1 {
429 blob, err := g.HashObject(content)
430 if err != nil {
431 return "", err
432 }
433 entries = append(entries, gitx.Entry{Mode: "100644", Type: "blob", SHA: blob, Name: parts[0]})
434 continue
435 }
436 if dirs[parts[0]] == nil {
437 dirs[parts[0]] = map[string][]byte{}
438 }
439 dirs[parts[0]][parts[1]] = content
440 }
441 for name, sub := range dirs {
442 sha, err := buildTree(g, sub)
443 if err != nil {
444 return "", err
445 }
446 entries = append(entries, gitx.Entry{Mode: "040000", Type: "tree", SHA: sha, Name: name})
447 }
448 return g.MkTree(entries)
449 }
450
451 // keyFileName derives a filesystem-safe name from the key itself. The filename
452 // is only an address; the .pub content stays authoritative.
453 func keyFileName(pub ssh.PublicKey) string {
454 sum := sha256.Sum256(pub.Marshal())
455 return hex.EncodeToString(sum[:])
456 }
457
458 // userNameFromPath extracts the user from "users/<name>/...".
459 func userNameFromPath(p string) string {
460 parts := strings.Split(p, "/")
461 if len(parts) < 3 || parts[0] != usersDir {
462 return ""
463 }
464 return parts[1]
465 }
466
467 func randomID() (string, error) {
468 b := make([]byte, 12)
469 if _, err := rand.Read(b); err != nil {
470 return "", err
471 }
472 return hex.EncodeToString(b), nil
473 }
474
475 // KeyDir is where a user's keys live inside the tree.
476 func KeyDir(name string) string { return filepath.Join(usersDir, name, keysDir) }