fix(groups): stop resync destroying the key it just restored

The fifth view-to-local write that drops a group's block-encryption key, and
the only one ever observed firing in the wild (TODO #123).

resyncGroup walks a raw Autobase view read-stream and dispatches on the key
prefix, then merges each `groups:` record over the local one. The view copy is
keyless by construction - appendGroupWithAvatarSplit strips the key on every
append - and the merge restored color, name, emoji, icon, joinedAt, members and
removedMembers from local while saying nothing about encryptionKey. So it wrote
a keyless record over a keyed one, after which the group reopens unencrypted on
the raw groupKey topic, stops syncing against keyed peers and mints invites with
no `enc=`.

Why the PR #231 audit could not see it: that audit searched for
`db.put(NS.groups`, and this site is `db.put(key, mergedGroup)` where `key` came
out of a stream. It never mentioned the namespace, never carried a tag and could
therefore never log a BLOCKED line, which is why six reproduction attempts and a
choke-point fix all left the symptom alive.

Why it kept coming back: resyncGroup is called from the UI, not the sync engine
- App.jsx:744 on group-joined and App.jsx:1346 on the TODO #124 keyless repair.
Every earlier reproduction drove the worklet directly, so the resync never ran.
The repair path is the cruel one: it restores the key and then immediately calls
the thing that destroys it.

The fix is in two parts, because carrying the key across would fix the instance
and leave the class untouched:

- the merge carries the local encryptionKey, exactly as the foregroundSync
  re-mirror already does
- a putStreamedRecord dispatcher routes every write whose key came from a
  read-stream, so the KEY decides the writer and a group record cannot be
  written raw by a caller that never thought about groups. Pure decisions
  isGroupRecordKey / groupIdFromRecordKey in src/lib/groupRecord.js.

Verified end to end against the real worklet under plain Node, with the pre-fix
source as the control so the result cannot be vacuous: create an encrypted
group, call resyncGroup once, read the record back. Pre-fix the key is gone and
the latch with it; with the fix both survive. 278 unit tests, 12 new - five on
the routing decision, seven scanning src/bare.js for the shape that hid, which
fail on the original write and pass on this one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HG8ayyquJuDMPSVLKDQKVh
This commit is contained in:
Your Name
2026-07-26 23:07:44 -05:00
co-authored by Claude Opus 5
parent f4ce2d55e9
commit bc5a3bfef3
4 changed files with 272 additions and 5 deletions
+33 -4
View File
@@ -20,6 +20,7 @@ const { SEEDER_PAIR_SCAN_TIMEOUT_MS } = require('./lib/seederPairTiming.js')
const {
resolveGroupEncryptionKey, resolveGroupEncryptedFlag, classifyKeylessGroup, resolvedPeerCount,
needsEncryptedLatchBackfill, isEncryptedButKeyless,
isGroupRecordKey, groupIdFromRecordKey,
} = require('./lib/groupRecord.js')
const { raceAppend, APPEND_TIMEOUT_MS } = require('./lib/appendTimeout.js')
const { shouldSwallowFault, parseConflictLog } = require('./lib/conflictSeatbelt.js')
@@ -1946,6 +1947,21 @@ async function putGroupRecord (groupId, value, tag) {
return next
}
// Write a local record whose key came out of a read-stream rather than being
// built from a literal namespace (TODO #123).
//
// This is the structural half of the fix. putGroupRecord only guards writes that
// remember to call it, and a stream-keyed write has no reason to: at that point
// a group record is just another row, and `db.put(key, value)` is the obvious
// thing to type. That is exactly how resyncGroup's view merge destroyed keys for
// months while an audit of the namespace found nothing. Routing every
// stream-keyed write through here makes the KEY decide the writer, so a group
// record cannot be written raw even by a caller that never thought about groups.
async function putStreamedRecord (key, value, tag) {
if (isGroupRecordKey(key)) return putGroupRecord(groupIdFromRecordKey(key), value, tag)
return db.put(key, value)
}
// A group is joined for this long with no peer-supplied membership before the
// never-synced heuristic calls it damaged. Long enough that a group whose other
// members are simply offline is not accused; short enough to still be useful.
@@ -5399,7 +5415,7 @@ async function resyncGroup (groupId) {
if (await isEventTombstoned(eventId, groupId)) continue
const existing = await db.get(key).catch(() => null)
if (existing?.value && sameExceptUpdatedAt(existing.value, value)) continue
await db.put(key, value)
await putStreamedRecord(key, value, 'resyncGroup:event')
changedEvents.push(value)
} else if (key.startsWith(NS.avatars)) {
// Repair missing avatar bytes. apply() mirrors avatars via mirrorToLocal
@@ -5408,8 +5424,8 @@ async function resyncGroup (groupId) {
// records with no local bytes — UI renders "?". Put-if-absent matches
// mirrorToLocal's avatar branch.
const existing = await db.get(key).catch(() => null)
if (!existing) await db.put(key, value)
} else if (key.startsWith('groups:')) {
if (!existing) await putStreamedRecord(key, value, 'resyncGroup:avatar')
} else if (isGroupRecordKey(key)) {
const existing = await db.get(key).catch(() => null)
const localJoinedAt = existing?.value?.joinedAt ?? 0
const viewUpdatedAt = value?.updatedAt ?? 0
@@ -5456,11 +5472,24 @@ async function resyncGroup (groupId) {
emoji: value.emoji || ev?.emoji,
icon: value.icon ?? ev?.icon,
joinedAt: ev?.joinedAt || value.joinedAt,
// FIFTH view-to-local write that dropped the local-only key, and the
// only one ever observed firing in the wild (TODO #123). The view copy
// is keyless by construction - appendGroupWithAvatarSplit strips the
// key on every append - so merging it over the local record silently
// disables the group's encryption unless the local key is carried
// across, exactly as the foregroundSync re-mirror does. This site is
// reached on every join (App.jsx:744) and on the TODO #124 keyless
// repair (App.jsx:1346), so it destroyed the key moments after the
// repair had restored it, which is why the banner kept coming back.
encryptionKey: ev?.encryptionKey || value.encryptionKey,
removedMembers: [...removedMap.values()],
members: splitMembers,
}
if (ev && sameExceptUpdatedAt(ev, mergedGroup)) continue
await db.put(key, mergedGroup)
// Through the dispatcher, not db.put: writing the key back above is
// belt, this is braces, and it means a future regression here logs a
// BLOCKED line naming this site instead of failing silently.
await putStreamedRecord(key, mergedGroup, 'resyncGroup:view-merge')
groupChanged = true
}
}
+39
View File
@@ -10,9 +10,45 @@
// local group-record write through a single guard. This is that guard's pure
// decision, split out so it is unit-testable (bare.js touches BareKit/Pear at
// load and cannot be required from tests). Same split as ownerGuard.js.
//
// A FIFTH site was found on 2026-07-26 and it is the one that actually fired in
// the wild - see isGroupRecordKey below for why the others' audit could not see
// it.
'use strict'
// The local namespace group records live in. Must match bare.js's `NS.groups`;
// the source-scan test in test/groupWriteChokePoint.test.js asserts it does.
const GROUP_KEY_PREFIX = 'groups:'
// Is this local-DB key a group record, i.e. one that MUST be written through
// putGroupRecord rather than db.put?
//
// This exists because of how the fifth key-dropping site hid. The other four
// dispatched on a typed op (`type === 'group'`) or built their key as
// `NS.groups + id`, so the PR #231 audit found them by searching for
// `db.put(NS.groups`. resyncGroup instead walks a raw Autobase view read-stream
// and dispatches on the KEY PREFIX, so its write reads `db.put(key, merged)` -
// textually invisible to that search, never tagged, and therefore never able to
// log a BLOCKED line. It was reached on every join and on the TODO #124 keyless
// repair, so it destroyed the key moments after the repair restored it.
//
// `groupMembers:` is a different namespace that shares the `group` stem and
// diverges at the sixth character, so it does not match and needs no special
// case. There is a test for exactly that, because it is the kind of thing a
// later prefix change would break silently.
function isGroupRecordKey (key) {
return typeof key === 'string' && key.startsWith(GROUP_KEY_PREFIX)
}
// The group id inside a group-record key. Returns null for anything that is not
// one, so a caller cannot accidentally address the whole namespace.
function groupIdFromRecordKey (key) {
if (!isGroupRecordKey(key)) return null
const id = key.slice(GROUP_KEY_PREFIX.length)
return id.length ? id : null
}
// Decide which encryptionKey a group-record write should actually persist.
//
// { key, blocked, reason }
@@ -137,6 +173,9 @@ function isEncryptedButKeyless (record) {
}
module.exports = {
GROUP_KEY_PREFIX,
isGroupRecordKey,
groupIdFromRecordKey,
resolveGroupEncryptionKey,
resolveGroupEncryptedFlag,
classifyKeylessGroup,
+42 -1
View File
@@ -3,7 +3,9 @@
// (bugfix/desktop-group-records)
const test = require('node:test')
const assert = require('node:assert/strict')
const { resolveGroupEncryptionKey } = require('../src/lib/groupRecord.js')
const {
resolveGroupEncryptionKey, isGroupRecordKey, groupIdFromRecordKey, GROUP_KEY_PREFIX,
} = require('../src/lib/groupRecord.js')
const KEY = 'a'.repeat(64)
const OTHER = 'b'.repeat(64)
@@ -56,3 +58,42 @@ test('back-filling a key onto a previously keyless record is allowed', () => {
assert.equal(r.key, KEY)
assert.equal(r.blocked, false)
})
// ── which keys must go through the guard (the fifth site, TODO #123) ──────
// resyncGroup walks a raw Autobase view read-stream and dispatches on the key
// prefix, so unlike the other four sites it never mentions the namespace it is
// writing. These are the decisions that let a stream-keyed write route itself.
test('a group-record key is recognised as one', () => {
assert.equal(isGroupRecordKey('groups:g0soe8x'), true)
assert.equal(groupIdFromRecordKey('groups:g0soe8x'), 'g0soe8x')
})
test('groupMembers: is a DIFFERENT namespace and must not match', () => {
// It shares the `group` stem and diverges at the sixth character. Nothing in
// the code special-cases that, so it is worth a test: a later rename that made
// the prefixes nest would silently route TODO #70's split member records
// through the group guard, which would then defend a key they never carry.
assert.equal(isGroupRecordKey('groupMembers:g0soe8x'), false)
assert.equal(groupIdFromRecordKey('groupMembers:g0soe8x'), null)
})
test('other namespaces in the same DB are left alone', () => {
// resyncGroup writes events and avatars through the same dispatcher, so these
// must fall through to a plain db.put rather than the group guard.
for (const k of ['events:2026-07-26:abc', 'avatars:deadbeef', 'members:g1:m1', 'profile']) {
assert.equal(isGroupRecordKey(k), false, k)
}
})
test('the namespace itself is not a group id', () => {
// Guards against a truncated key addressing the whole prefix.
assert.equal(groupIdFromRecordKey(GROUP_KEY_PREFIX), null)
})
test('tolerates non-string keys', () => {
for (const k of [null, undefined, 42, {}]) {
assert.equal(isGroupRecordKey(k), false)
assert.equal(groupIdFromRecordKey(k), null)
}
})
+158
View File
@@ -0,0 +1,158 @@
// TODO #123 - the choke point is only a choke point if nothing can write past it.
//
// PR #231 routed sixteen local group-record writes through putGroupRecord and
// verified the result by searching for `db.put(NS.groups`. That search returned
// zero, and the conclusion drawn was "every write is guarded". It was wrong: a
// seventeenth write existed in resyncGroup as `db.put(key, mergedGroup)`, where
// `key` came from an Autobase view read-stream and was dispatched on its prefix.
// Textually it looks like any other mirror write, so the audit could not see it,
// it carried no tag, and it could therefore never log a BLOCKED line. It ran on
// every join and on the #124 keyless repair, and it is the site that actually
// destroyed keys in the wild.
//
// A unit test on a pure decision cannot catch that class, because the defect is
// which function was called, not what the function computed. So this test reads
// src/bare.js and enforces the invariant directly on the source. It is the check
// that would have failed in July.
//
// What it can and cannot see, stated plainly: it catches a raw db.put whose key
// is built from NS.groups, and a raw db.put with a variable key inside a
// function that prefix-dispatches on the group namespace (the resyncGroup
// shape). It cannot see a variable key in a function that never mentions the
// namespace at all - nothing short of running the code can. That residue is
// covered by the guard itself, which is why the fix carries the key across AND
// routes through putGroupRecord rather than relying on either alone.
// (bugfix/resync-drops-encryption-key)
const test = require('node:test')
const assert = require('node:assert/strict')
const fs = require('node:fs')
const path = require('node:path')
const { GROUP_KEY_PREFIX } = require('../src/lib/groupRecord.js')
const SRC = fs.readFileSync(path.join(__dirname, '..', 'src', 'bare.js'), 'utf8')
// The one function allowed to write the namespace. Everything else must call it.
const CHOKE_POINT = 'putGroupRecord'
// The dispatcher for keys that come out of a read-stream: it is the only other
// place a bare db.put with a variable key is correct, because it is the thing
// deciding, from the key, whether the write is a group record at all.
const DISPATCHER = 'putStreamedRecord'
// Split the source into top-level function bodies. bare.js declares every
// function at column zero, so this is exact rather than heuristic: a body runs
// from its `function` line to the next one.
function topLevelFunctions (src) {
const lines = src.split('\n')
const starts = []
for (let i = 0; i < lines.length; i++) {
const m = /^(?:async\s+)?function\s+([A-Za-z0-9_$]+)\s*\(/.exec(lines[i])
if (m) starts.push({ name: m[1], line: i })
}
return starts.map((s, i) => ({
name: s.name,
startLine: s.line + 1,
body: lines.slice(s.line, i + 1 < starts.length ? starts[i + 1].line : lines.length).join('\n'),
}))
}
// Pull the first argument of every `db.put(...)` in a body, depth-aware so that
// nested calls and object literals do not truncate it, and newline-tolerant
// because several call sites wrap.
function dbPutKeyArgs (body) {
const out = []
const needle = 'db.put('
let idx = body.indexOf(needle)
while (idx !== -1) {
let depth = 1
let i = idx + needle.length
const from = i
while (i < body.length && depth > 0) {
const c = body[i]
if (c === '(' || c === '[' || c === '{') depth++
else if (c === ')' || c === ']' || c === '}') depth--
else if (c === ',' && depth === 1) break
i++
}
out.push({ arg: body.slice(from, i).trim(), at: idx })
idx = body.indexOf(needle, idx + needle.length)
}
return out
}
const FUNCTIONS = topLevelFunctions(SRC)
test('bare.js NS.groups matches the prefix the decision module uses', () => {
// isGroupRecordKey hardcodes the prefix so it stays pure. If bare.js ever
// renames the namespace, every routing decision here silently stops matching
// and group writes quietly go raw again.
const m = /groups:\s*'([^']+)'/.exec(SRC)
assert.ok(m, 'could not find the NS.groups declaration in bare.js')
assert.equal(m[1], GROUP_KEY_PREFIX)
})
test('no function builds a group key and writes it with db.put', () => {
const offenders = []
for (const fn of FUNCTIONS) {
if (fn.name === CHOKE_POINT) continue
for (const { arg } of dbPutKeyArgs(fn.body)) {
if (/\bNS\.groups\b/.test(arg)) offenders.push(`${fn.name}: db.put(${arg}, …)`)
}
}
assert.deepEqual(offenders, [], `raw group-record writes must call ${CHOKE_POINT}()`)
})
test('no function that dispatches on the group prefix writes with a bare db.put', () => {
// This is the resyncGroup shape, and the one the original audit was blind to:
// the key is a variable, so the write mentions no namespace at all, while the
// branch it sits in is guarded by a group-prefix test a few lines above.
const dispatches = /isGroupRecordKey\s*\(|startsWith\s*\(\s*NS\.groups|startsWith\s*\(\s*'groups:'/
const bareIdentifier = /^[A-Za-z_$][A-Za-z0-9_$]*$/
const offenders = []
for (const fn of FUNCTIONS) {
if (fn.name === CHOKE_POINT || fn.name === DISPATCHER) continue
if (!dispatches.test(fn.body)) continue
for (const { arg } of dbPutKeyArgs(fn.body)) {
if (bareIdentifier.test(arg)) offenders.push(`${fn.name}: db.put(${arg}, …)`)
}
}
assert.deepEqual(offenders, [],
'a function that prefix-dispatches on group keys must route its writes through ' +
`${CHOKE_POINT}() - a bare db.put(key, …) there cannot tell a group record from any other row`)
})
test('the stream-key dispatcher sends group keys to the choke point', () => {
// The dispatcher is the only bare db.put with a variable key that is allowed,
// so it has to actually do the routing that earns the exemption.
const fn = FUNCTIONS.find(f => f.name === DISPATCHER)
assert.ok(fn, `${DISPATCHER} not found - rename? update this test`)
assert.match(fn.body, /isGroupRecordKey\s*\(\s*key\s*\)/, 'it must decide from the key')
assert.match(fn.body, new RegExp(`${CHOKE_POINT}\\(`), 'and send group records to the guard')
})
test('resyncGroup routes its group-record write through the dispatcher', () => {
// Positive assertion, so deleting the call is a failure rather than a silent
// pass of the two negative tests above.
const fn = FUNCTIONS.find(f => f.name === 'resyncGroup')
assert.ok(fn, 'resyncGroup not found - rename? update this test')
assert.match(fn.body, new RegExp(`${DISPATCHER}\\(`),
'resyncGroup merges the keyless view record over the local one; it must write it through the dispatcher')
})
test('resyncGroup carries the local encryptionKey across the view merge', () => {
// Belt as well as braces. The guard alone would preserve the key, but it would
// also log a BLOCKED line on every single resync, turning the one diagnostic
// that names a culprit into noise. Carrying the key means the guard stays
// silent unless something is genuinely wrong.
const fn = FUNCTIONS.find(f => f.name === 'resyncGroup')
assert.match(fn.body, /encryptionKey:\s*ev\?\.encryptionKey/,
'the merged record must take the local key, or every resync trips the guard')
})
test('the view append still strips the key it is stripping for', () => {
// The whole hazard rests on this: view records are keyless, so any merge of a
// view record over a local one is a key drop unless it says otherwise. If this
// ever stops being true the reasoning above changes completely.
const fn = FUNCTIONS.find(f => f.name === 'appendGroupWithAvatarSplit')
assert.ok(fn, 'appendGroupWithAvatarSplit not found')
assert.match(fn.body, /encryptionKey:\s*_ek/, 'the key must be destructured out before the append')
})