Sign in

Search issue titles and bodies, change requests and comments. Private repositories you cannot read are not included.
vole
Clone

Read only, no credentials:

https://rodent.software/vole.git

Write over SSH with a registered key:

git clone ssh://git@rodent.software:2222/vole.git

A free and open-source forge that lives in your Git repo.

Replace the git-backed roster with a trusted keys directory

197ecaa29d992ecce2b6078803f3512f565f3672 · Keenan Nemetz · 18d ago · parent a5f19b3c 

Replace the git-backed roster with a trusted keys directory

The roster was 476 lines of CAS retries, tree building and snapshot reloading to
store what amounts to username -> authorized_keys lines. Two of the three fields
in meta.json were unused: ACLs key on username, never on id, and nothing ever set
disabled. The git version was paying for history and tamper evidence that
nothing consumed.

Now:

  data/trusted_keys/keenan.keys    # one authorized_keys line per device

This is the AuthorizedKeysFile convention, so ssh-keygen -lf, ansible, rsync and
an editor all work on it, and dropping a key in by hand is a supported way to add
a user because the running instance re-reads the directory every five seconds.
Adds are O_APPEND, so two processes cannot lose each other's writes; removes are
read-modify-rename and assume a single operator. A file that does not parse, or
the same key under two names, still refuses startup rather than silently changing
who can authenticate.

What the directory could not provide on its own is an audit trail, so key changes
append a line to data/audit.log: timestamp, action, user, and key fingerprint.
That is the git log we deleted, at about 15 lines instead of a commit graph. If a
user must be blocked rather than removed, their keys are removed; there is no
disabled flag until something needs one.

Identity is now ~200 lines instead of 476, and the lost-update class of bug that
the CAS loop made possible is gone with the loop.

Verified by hand: fresh init shows data/ only holding audit.log, index.db, repos
and trusted_keys; a key appended to the file by hand starts working within the
poll interval without a restart; removing a key by CLI stops that key
authenticating; audit.log records both.

internal/identity/identity.go +150−312

@@ -1,24 +1,19 @@
1 // Package identity keeps the instance's user roster in git, inside a dedicated
2 // system repository.
1 // Package identity is the instance's user roster: one file of SSH public keys
2 // per user, in a trusted-keys directory.
3 3 //
4 // The directory listing is the table: users/<name>/keys/<hex>.pub. Everything
5 // stored here is public material, so a copy of the system repo is a complete
6 // backup that leaks nothing. Writes are compare-and-swap commits; reads come
7 // from an in-memory snapshot refreshed when the ref moves, because the SSH auth
8 // callback is on the hot path and must be a map lookup, not a git invocation.
4 // data/trusted_keys/keenan.keys # one authorized_keys line per device
9 5 //
10 // The roster lives on refs/heads/main on purpose. A custom namespace such as
11 // refs/vole/users is invisible to a plain `git clone`, which only mirrors
12 // refs/heads/*, so the obvious backup command would have silently produced an
13 // instance with no users. The system repo is never pushed to by clients, so it
14 // gains nothing from a reserved namespace.
6 // This is the same shape sshd uses for AuthorizedKeysFile, so the tools people
7 // already reach for work here: ssh-keygen -lf, ansible, rsync, an editor. Nothing
8 // stored is secret, which is what lets the directory be copied or put under
9 // configuration management without leaking anything.
10 //
11 // Adds are appends, so two processes cannot lose each other's writes. Removes are
12 // read-modify-rename and assume a single operator, which is the deployment this
13 // is built for.
15 14 package identity
16 15
17 16 import (
18 "crypto/rand"
19 "crypto/sha256"
20 "encoding/hex"
21 "encoding/json"
22 17 "fmt"
23 18 "os"
24 19 "path/filepath"
@@ -29,40 +24,30 @@ import (
29 24 "time"
30 25
31 26 "golang.org/x/crypto/ssh"
32
33 "github.com/rodent-software/vole/internal/gitx"
34 27 )
35 28
36 29 const (
37 UsersRef = "refs/heads/main"
38 usersDir = "users"
39 metaFile = "meta.json"
40 keysDir = "keys"
41 maxCAS = 64
42 DirName = "system.git"
30 keysSuffix = ".keys"
31 header = "# vole: SSH public keys for %s, one per device (authorized_keys format).\n"
43 32 )
44 33
45 34 type User struct {
46 Name string
47 ID string
48 Created time.Time
49 Disabled bool
50 Keys []Key
35 Name string
36 Keys []Key
51 37 }
52 38
53 39 type Key struct {
54 Fingerprint string // ssh.FingerprintSHA256 of the key
40 Fingerprint string
55 41 Type string
56 42 Comment string
57 Line string // authorized_keys-format line
58 Added time.Time
43 Line string
59 44 }
60 45
61 // Snapshot is an immutable view of the roster, swapped atomically.
46 // Snapshot is an immutable view of the roster, swapped atomically so the SSH
47 // auth callback is a map lookup rather than a file read.
62 48 type Snapshot struct {
63 49 Users map[string]*User
64 50 ByFingerprint map[string]string
65 SHA string
66 51 }
67 52
68 53 func (s *Snapshot) Lookup(fingerprint string) (string, bool) {
@@ -86,149 +71,85 @@ func (s *Snapshot) Names() []string {
86 71 }
87 72
88 73 type Store struct {
89 Dir string
90 Git *gitx.Repo
91 snap atomic.Pointer[Snapshot]
92 mu sync.Mutex // serialises local writers
74 Dir string
75 // AuditPath, when set, receives one line per key change: the audit trail a
76 // git-backed roster used to provide for free.
77 AuditPath string
78 snap atomic.Pointer[Snapshot]
79 mu sync.Mutex
93 80 }
94 81
95 // Init creates the system repo with an empty roster.
96 func Init(dir string) error {
97 if _, err := os.Stat(dir); err == nil {
98 return nil
99 }
100 if err := gitx.InitBare(dir, "main"); err != nil {
101 return err
102 }
103 s, err := Open(dir)
104 if err != nil {
105 return err
106 }
107 return s.writeTree(map[string][]byte{
108 usersDir + "/.keep": []byte("the users tree: one directory per user\n"),
109 }, "users: initialise empty roster", "vole")
110 }
82 // Init creates the trusted-keys directory if it is missing.
83 func Init(dir string) error { return os.MkdirAll(dir, 0o755) }
111 84
112 // Open loads an existing system repo. A missing or unreadable roster is a hard
113 // error: booting with an empty roster would silently mean "nobody can
114 // authenticate", and the diagnosis would happen at an inconvenient hour.
85 // Open loads the roster. An unreadable or unparsable file is a hard error:
86 // booting with a partial roster would silently change who can authenticate.
115 87 func Open(dir string) (*Store, error) {
116 g := gitx.Open(dir)
117 if !g.Exists() {
88 st, err := os.Stat(dir)
89 if err != nil || !st.IsDir() {
118 90 return nil, fmt.Errorf("identity: %s is missing; run vole init", dir)
119 91 }
120 s := &Store{Dir: dir, Git: g}
92 s := &Store{Dir: dir}
121 93 if err := s.Reload(); err != nil {
122 94 return nil, err
123 95 }
124 96 return s, nil
125 97 }
126 98
127 // Reload re-reads the roster from the ref.
128 99 func (s *Store) Reload() error {
129 sha, err := s.Git.RevParseRaw(UsersRef)
100 entries, err := os.ReadDir(s.Dir)
130 101 if err != nil {
131 102 return err
132 103 }
133 snap := &Snapshot{Users: map[string]*User{}, ByFingerprint: map[string]string{}, SHA: sha}
134 if sha == "" {
135 s.snap.Store(snap)
136 return nil
137 }
138 // A ref pointing at anything but a commit means the roster is corrupt. Left
139 // unchecked, `ls-tree` on a blob prints nothing and the instance would boot
140 // with an empty roster — meaning nobody can authenticate.
141 typ, err := s.Git.ObjectType(sha)
142 if err != nil {
143 return fmt.Errorf("identity: read %s: %w", UsersRef, err)
144 }
145 if typ != "commit" {
146 return fmt.Errorf("identity: %s points at a %s, want a commit", UsersRef, typ)
147 }
148 entries, err := s.Git.LsTreeR(sha)
149 if err != nil {
150 return fmt.Errorf("identity: read roster: %w", err)
151 }
104 snap := &Snapshot{Users: map[string]*User{}, ByFingerprint: map[string]string{}}
152 105 for _, e := range entries {
153 if e.Type != "blob" || !strings.HasSuffix(e.Name, ".pub") {
106 if e.IsDir() || !strings.HasSuffix(e.Name(), keysSuffix) {
154 107 continue
155 108 }
156 name := userNameFromPath(e.Name)
157 if name == "" {
109 name := strings.TrimSuffix(e.Name(), keysSuffix)
110 if strings.HasPrefix(e.Name(), ".") {
158 111 continue
159 112 }
160 data, err := s.Git.BlobBySHA(e.SHA)
161 if err != nil {
162 return fmt.Errorf("identity: read key %s: %w", e.Name, err)
113 if err := ValidateName(name); err != nil {
114 return fmt.Errorf("identity: file %q: %w", e.Name(), err)
163 115 }
164 pub, comment, _, rest, err := ssh.ParseAuthorizedKey(data)
165 if err != nil {
166 return fmt.Errorf("identity: %s: unparsable key: %w", e.Name, err)
167 }
168 _ = rest
169 u := snap.Users[name]
170 if u == nil {
171 u = &User{Name: name}
172 snap.Users[name] = u
173 }
174 fp := ssh.FingerprintSHA256(pub)
175 u.Keys = append(u.Keys, Key{
176 Fingerprint: fp,
177 Type: pub.Type(),
178 Comment: comment,
179 Line: strings.TrimSpace(string(data)),
180 })
181 snap.ByFingerprint[fp] = name
182 }
183 // Metadata files.
184 for _, e := range entries {
185 if e.Type != "blob" || !strings.HasSuffix(e.Name, "/"+metaFile) {
186 continue
187 }
188 name := userNameFromPath(e.Name)
189 if name == "" {
190 continue
191 }
192 data, err := s.Git.BlobBySHA(e.SHA)
116 data, err := os.ReadFile(filepath.Join(s.Dir, e.Name()))
193 117 if err != nil {
194 118 return err
195 119 }
196 var m struct {
197 ID string `json:"id"`
198 Created time.Time `json:"created"`
199 Disabled bool `json:"disabled"`
200 }
201 if err := json.Unmarshal(data, &m); err != nil {
202 return fmt.Errorf("identity: %s/%s: %w", name, metaFile, err)
203 }
204 u := snap.Users[name]
205 if u == nil {
206 u = &User{Name: name}
207 snap.Users[name] = u
120 u := &User{Name: name}
121 for i, line := range strings.Split(string(data), "\n") {
122 line = strings.TrimSpace(line)
123 if line == "" || strings.HasPrefix(line, "#") {
124 continue
125 }
126 pub, comment, _, _, err := ssh.ParseAuthorizedKey([]byte(line))
127 if err != nil {
128 return fmt.Errorf("identity: %s line %d: %w", e.Name(), i+1, err)
129 }
130 fp := ssh.FingerprintSHA256(pub)
131 if owner, ok := snap.ByFingerprint[fp]; ok {
132 return fmt.Errorf("identity: key %s appears under both %s and %s", fp, owner, name)
133 }
134 u.Keys = append(u.Keys, Key{
135 Fingerprint: fp,
136 Type: pub.Type(),
137 Comment: comment,
138 Line: line,
139 })
140 snap.ByFingerprint[fp] = name
208 141 }
209 u.ID, u.Created, u.Disabled = m.ID, m.Created, m.Disabled
210 }
211 // Order keys for stable output.
212 for _, u := range snap.Users {
213 142 sort.Slice(u.Keys, func(i, j int) bool { return u.Keys[i].Fingerprint < u.Keys[j].Fingerprint })
143 snap.Users[name] = u
214 144 }
215 145 s.snap.Store(snap)
216 146 return nil
217 147 }
218 148
219 // Snapshot returns the current roster view. Cheap and lock-free.
220 149 func (s *Store) Snapshot() *Snapshot { return s.snap.Load() }
221 150
222 // Lookup resolves a key fingerprint to a username.
223 151 func (s *Store) Lookup(fingerprint string) (string, bool) {
224 name, ok := s.Snapshot().Lookup(fingerprint)
225 if !ok {
226 return "", false
227 }
228 if u, ok := s.Snapshot().Users[name]; ok && u.Disabled {
229 return "", false
230 }
231 return name, true
152 return s.Snapshot().Lookup(fingerprint)
232 153 }
233 154
234 155 func ValidateName(name string) error {
@@ -244,8 +165,9 @@ func ValidateName(name string) error {
244 165 return nil
245 166 }
246 167
247 // Create adds a user with no keys. They cannot authenticate until a key is
248 // added, which is the intended order of operations.
168 func (s *Store) path(name string) string { return filepath.Join(s.Dir, name+keysSuffix) }
169
170 // Create adds an empty key file: a user who exists but cannot authenticate yet.
249 171 func (s *Store) Create(name string) error {
250 172 if err := ValidateName(name); err != nil {
251 173 return err
@@ -253,27 +175,26 @@ func (s *Store) Create(name string) error {
253 175 if _, ok := s.Snapshot().Users[name]; ok {
254 176 return fmt.Errorf("identity: user %q already exists", name)
255 177 }
256 id, err := randomID()
178 f, err := os.OpenFile(s.path(name), os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o644)
257 179 if err != nil {
180 if os.IsExist(err) {
181 return fmt.Errorf("identity: user %q already exists", name)
182 }
258 183 return err
259 184 }
260 meta, err := json.MarshalIndent(struct {
261 ID string `json:"id"`
262 Created time.Time `json:"created"`
263 Disabled bool `json:"disabled"`
264 }{id, time.Now().UTC(), false}, "", " ")
265 if err != nil {
185 defer f.Close()
186 if _, err := fmt.Fprintf(f, header, name); err != nil {
266 187 return err
267 188 }
268 return s.apply(func(files map[string][]byte) error {
269 files[fmt.Sprintf("%s/%s/%s", usersDir, name, metaFile)] = append(meta, '\n')
270 return nil
271 }, fmt.Sprintf("users: create %s", name), "vole")
189 s.audit("identity.create", name, "")
190 return s.Reload()
272 191 }
273 192
274 193 // AddKey registers an SSH public key for a user and returns its fingerprint.
275 func (s *Store) AddKey(name, authorizedKey, addedBy string) (string, error) {
276 pub, _, _, _, err := ssh.ParseAuthorizedKey([]byte(strings.TrimSpace(authorizedKey)))
194 // The write is an append, so it cannot lose a concurrent add.
195 func (s *Store) AddKey(name, authorizedKey, actor string) (string, error) {
196 line := strings.TrimSpace(authorizedKey)
197 pub, comment, _, _, err := ssh.ParseAuthorizedKey([]byte(line))
277 198 if err != nil {
278 199 return "", fmt.Errorf("identity: not an SSH public key: %w", err)
279 200 }
@@ -284,193 +205,110 @@ func (s *Store) AddKey(name, authorizedKey, addedBy string) (string, error) {
284 205 if _, ok := s.Snapshot().Users[name]; !ok {
285 206 return "", fmt.Errorf("identity: no such user %q", name)
286 207 }
287 line := strings.TrimSpace(authorizedKey) + "\n"
288 path := fmt.Sprintf("%s/%s/%s/%s.pub", usersDir, name, keysDir, keyFileName(pub))
289 err = s.apply(func(files map[string][]byte) error {
290 files[path] = []byte(line)
291 return nil
292 }, fmt.Sprintf("users: add key for %s (%s)", name, fp), addedBy)
208
209 f, err := os.OpenFile(s.path(name), os.O_APPEND|os.O_WRONLY, 0o644)
293 210 if err != nil {
294 211 return "", err
295 212 }
213 if _, err := fmt.Fprintf(f, "%s\n", line); err != nil {
214 f.Close()
215 return "", err
216 }
217 if err := f.Close(); err != nil {
218 return "", err
219 }
220 s.audit("identity.add-key", name, fp+" "+comment)
221 if err := s.Reload(); err != nil {
222 return "", err
223 }
296 224 return fp, nil
297 225 }
298 226
299 // RemoveKey revokes a key. Recovery is an operator adding a new key, which is
300 // why there is no reset flow to build.
301 func (s *Store) RemoveKey(name, fingerprint, removedBy string) error {
227 // RemoveKey revokes a key by rewriting the file without it. Read-modify-rename,
228 // so it assumes a single operator process.
229 func (s *Store) RemoveKey(name, fingerprint, actor string) error {
230 s.mu.Lock()
231 defer s.mu.Unlock()
232
302 233 u, ok := s.Snapshot().Users[name]
303 234 if !ok {
304 235 return fmt.Errorf("identity: no such user %q", name)
305 236 }
306 var target *Key
307 for i := range u.Keys {
308 if u.Keys[i].Fingerprint == fingerprint {
309 target = &u.Keys[i]
237 found := false
238 for _, k := range u.Keys {
239 if k.Fingerprint == fingerprint {
240 found = true
310 241 break
311 242 }
312 243 }
313 if target == nil {
244 if !found {
314 245 return fmt.Errorf("identity: %s has no key %s", name, fingerprint)
315 246 }
316 pub, _, _, _, err := ssh.ParseAuthorizedKey([]byte(target.Line))
317 if err != nil {
318 return err
319 }
320 path := fmt.Sprintf("%s/%s/%s/%s.pub", usersDir, name, keysDir, keyFileName(pub))
321 return s.apply(func(files map[string][]byte) error {
322 if _, ok := files[path]; !ok {
323 return fmt.Errorf("identity: key file vanished: %s", path)
324 }
325 delete(files, path)
326 return nil
327 }, fmt.Sprintf("users: revoke key for %s (%s)", name, fingerprint), removedBy)
328 }
329 247
330 // apply is the single write path: read the tree, mutate a flat path map, write
331 // it back with a compare-and-swap ref update, then refresh the snapshot.
332 func (s *Store) apply(mutate func(map[string][]byte) error, message, actor string) error {
333 s.mu.Lock()
334 defer s.mu.Unlock()
335
336 // The read, the mutation and the compare must all see the same ref value.
337 // Reading the tree once outside this loop and then CAS-ing against a fresh
338 // old value silently discards a concurrent writer's commit, because the CAS
339 // succeeds while the tree content is stale.
340 var lastErr error
341 for attempt := 0; attempt < maxCAS; attempt++ {
342 old, err := s.Git.RevParseRaw(UsersRef)
343 if err != nil {
344 return err
345 }
346 files, err := s.readFlat(old)
347 if err != nil {
348 return err
349 }
350 if err := mutate(files); err != nil {
351 return err
352 }
353 tree, err := buildTree(s.Git, files)
354 if err != nil {
355 return err
356 }
357 commit, err := s.Git.CommitTree(tree, old, withActor(message, actor))
358 if err != nil {
359 return err
360 }
361 ok, err := s.Git.UpdateRefCAS(UsersRef, commit, old)
362 if err != nil {
363 return err
364 }
365 if ok {
366 return s.Reload()
367 }
368 lastErr = fmt.Errorf("identity: lost %d ref race(s)", attempt+1)
369 time.Sleep(time.Duration(attempt+1) * 5 * time.Millisecond)
370 }
371 return lastErr
372 }
373
374 func (s *Store) readFlat(sha string) (map[string][]byte, error) {
375 out := map[string][]byte{}
376 if sha == "" {
377 return out, nil
378 }
379 if typ, err := s.Git.ObjectType(sha); err != nil {
380 return nil, err
381 } else if typ != "commit" {
382 return nil, fmt.Errorf("identity: %s points at a %s, want a commit", UsersRef, typ)
383 }
384 entries, err := s.Git.LsTreeR(sha)
248 p := s.path(name)
249 data, err := os.ReadFile(p)
385 250 if err != nil {
386 return nil, err
251 return err
387 252 }
388 for _, e := range entries {
389 if e.Type != "blob" {
253 var kept []string
254 for _, line := range strings.Split(string(data), "\n") {
255 trimmed := strings.TrimSpace(line)
256 switch {
257 case trimmed == "":
390 258 continue
259 case strings.HasPrefix(trimmed, "#"):
260 kept = append(kept, line)
261 default:
262 pub, _, _, _, err := ssh.ParseAuthorizedKey([]byte(trimmed))
263 if err != nil {
264 return fmt.Errorf("identity: %s: %w", p, err)
265 }
266 if ssh.FingerprintSHA256(pub) != fingerprint {
267 kept = append(kept, line)
268 }
391 269 }
392 data, err := s.Git.BlobBySHA(e.SHA)
393 if err != nil {
394 return nil, err
395 }
396 out[e.Name] = data
397 270 }
398 return out, nil
271 if err := writeAtomic(p, strings.Join(kept, "\n")+"\n"); err != nil {
272 return err
273 }
274 s.audit("identity.remove-key", name, fingerprint)
275 return s.Reload()
399 276 }
400 277
401 // writeTree is the one-shot path used by Init, where the ref cannot exist yet.
402 func (s *Store) writeTree(files map[string][]byte, message, actor string) error {
403 tree, err := buildTree(s.Git, files)
278 // writeAtomic replaces path via a temp file in the same directory, so a reader
279 // never observes a half-written roster.
280 func writeAtomic(path, content string) error {
281 dir := filepath.Dir(path)
282 f, err := os.CreateTemp(dir, ".vole-keys-*")
404 283 if err != nil {
405 284 return err
406 285 }
407 commit, err := s.Git.CommitTree(tree, "", withActor(message, actor))
408 if err != nil {
286 tmp := f.Name()
287 defer os.Remove(tmp)
288 if _, err := f.WriteString(content); err != nil {
289 f.Close()
409 290 return err
410 291 }
411 _, err = s.Git.UpdateRefCAS(UsersRef, commit, "")
412 return err
413 }
414
415 func withActor(message, actor string) string {
416 if strings.TrimSpace(actor) == "" {
417 actor = "unknown"
418 }
419 return message + "\n\nBy: " + actor
420 }
421
422 // buildTree writes a nested tree from flat paths.
423 func buildTree(g *gitx.Repo, files map[string][]byte) (string, error) {
424 var entries []gitx.Entry
425 dirs := map[string]map[string][]byte{}
426 for path, content := range files {
427 parts := strings.SplitN(path, "/", 2)
428 if len(parts) == 1 {
429 blob, err := g.HashObject(content)
430 if err != nil {
431 return "", err
432 }
433 entries = append(entries, gitx.Entry{Mode: "100644", Type: "blob", SHA: blob, Name: parts[0]})
434 continue
435 }
436 if dirs[parts[0]] == nil {
437 dirs[parts[0]] = map[string][]byte{}
438 }
439 dirs[parts[0]][parts[1]] = content
292 if err := f.Close(); err != nil {
293 return err
440 294 }
441 for name, sub := range dirs {
442 sha, err := buildTree(g, sub)
443 if err != nil {
444 return "", err
445 }
446 entries = append(entries, gitx.Entry{Mode: "040000", Type: "tree", SHA: sha, Name: name})
295 if err := os.Chmod(tmp, 0o644); err != nil {
296 return err
447 297 }
448 return g.MkTree(entries)
298 return os.Rename(tmp, path)
449 299 }
450 300
451 // keyFileName derives a filesystem-safe name from the key itself. The filename
452 // is only an address; the .pub content stays authoritative.
453 func keyFileName(pub ssh.PublicKey) string {
454 sum := sha256.Sum256(pub.Marshal())
455 return hex.EncodeToString(sum[:])
456 }
457
458 // userNameFromPath extracts the user from "users/<name>/...".
459 func userNameFromPath(p string) string {
460 parts := strings.Split(p, "/")
461 if len(parts) < 3 || parts[0] != usersDir {
462 return ""
301 // audit appends one line describing a roster change. Best effort: failing to
302 // write the log must not fail an otherwise valid key change.
303 func (s *Store) audit(action, user, detail string) {
304 if s.AuditPath == "" {
305 return
463 306 }
464 return parts[1]
465 }
466
467 func randomID() (string, error) {
468 b := make([]byte, 12)
469 if _, err := rand.Read(b); err != nil {
470 return "", err
307 f, err := os.OpenFile(s.AuditPath, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0o644)
308 if err != nil {
309 return
471 310 }
472 return hex.EncodeToString(b), nil
311 fmt.Fprintf(f, "%s\t%s\t%s\t%s\n",
312 time.Now().UTC().Format(time.RFC3339), action, user, detail)
313 f.Close()
473 314 }
474
475 // KeyDir is where a user's keys live inside the tree.
476 func KeyDir(name string) string { return filepath.Join(usersDir, name, keysDir) }