mirror of
https://github.com/vitorpamplona/amethyst.git
synced 2026-10-30 07:37:38 +00:00
Claude
9afa8127da
Quartz moves to its own repo, the way the crypto primitives moved to crypto-kmp. Every module depends on com.vitorpamplona.quartz:quartz 2.0.0 (libs.quartz) instead of :quartz, and -PquartzDir=../quartz builds a local checkout instead of the artifact, for every module and for quic's own quartz dependency (which replaces the root build's substitution of the published quartz with :quartz). What moved with quartz: its CI jobs (lint tasks, test-quartz-linux-native, test-quartz-ios), the Maven Central publish step of the release workflow, jitpack.yml, tools/unicode-nfkc, the nostr-expert and event-store-semantics skills, the agent-skill check, and the README's library usage docs. What stays here: - The four quartz client tests that need a real Ktor socket relay (TrustNetworkSync*, GiantReqStreamTest, BulkDownloadBenchmark) move to geode's tests; geode forwards -PprodRelayBench for the latter. quartz's other relay-client tests now use a test relay of its own, which ends quartz's test dependency on geode. - The benchmark module's resources (previously read from quartz's androidDeviceTest resources) move into benchmark/; relayBench --real reads the corpus from there. - NoteComposeKindCoverageTest reads EventFactory.kt from quartz's JVM sources jar, or from the -PquartzDir checkout, passed by commonsUI's build. - commons and commonsUI declare crypto-kmp (and commons mls-kmp), which they import directly. Through quartz's api alone, a -PcryptoKmpDir checkout never reached the common metadata compile. - The catalog drops what only quartz used (kchesslib, negentropy, jackson-databind, sqlite-bundled, secp256k1-kmp common, compose runtime-annotation, the maven-publish plugin). agent_config_check checks a skill's quartz/... paths against a ../quartz checkout when there is one and skips them otherwise. AGENTS.md, the skills, CONTRIBUTING, BUILDING, RELEASE_OPS, PLANS and the READMEs point at the new repo. Needs quartz 2.0.0 (and crypto-kmp 1.0.0) on Maven Central before CI can resolve it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DEkJaDWYrcTbGvT8JLQCrG
99 lines
5.0 KiB
Markdown
99 lines
5.0 KiB
Markdown
# LocalCache: the in-memory event store
|
|
|
|
`commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/model/cache/`:
|
|
|
|
- `LocalCache.kt` — `object LocalCache : EventCache()`, the process-wide instance both app modules use.
|
|
- `EventCache.kt` — all the logic (`open class EventCache : ILocalCache, ICacheProvider, Dao`).
|
|
It is a class so a test can build a cache of its own instead of sharing the singleton.
|
|
|
|
## Structures (fields of `EventCache`)
|
|
|
|
```kotlin
|
|
val users = LargeSoftCache<HexKey, User>()
|
|
val notes = LargeSoftCache<HexKey, Note>()
|
|
val addressables = LargeSoftCache<Address, AddressableNote>()
|
|
|
|
val chatroomList = LargeCache<HexKey, ChatroomList>()
|
|
val publicChatChannels = LargeCache<HexKey, PublicChatChannel>()
|
|
val liveChatChannels = LargeCache<Address, LiveActivitiesChannel>()
|
|
// … ephemeralChannels, geohashChannels, relayGroupChannels, concordChannels
|
|
val deletionIndex = DeletionIndex()
|
|
val observables = FilterIndex<Observable>()
|
|
val pruner = CachePruner(this)
|
|
val search = CacheSearch(this)
|
|
val live: LocalCacheFlow = LocalCacheFlow()
|
|
```
|
|
|
|
`LargeSoftCache` (`LargeSoftCache.kt`) holds **weak references** (a `ConcurrentSkipListMap` of weak
|
|
refs on JVM/Android). A `Note` or `User` nobody holds strongly can be collected, which is why every
|
|
account state object keeps its note in a field. Channel maps are plain `LargeCache` (quartz `utils/cache/LargeCache.kt`; see
|
|
quartz's `nostr-expert` skill).
|
|
|
|
## Insertion path
|
|
|
|
```
|
|
relay EVENT
|
|
└─ CacheClientConnector (commons/src/jvmAndroid/.../relayClient/) → cache.justConsume(event, relay, false)
|
|
├─ already deleted? → re-send the deletion to that relay, drop the event
|
|
├─ addressable and we hold a newer version? → re-send ours to that relay
|
|
└─ justConsumeAndUpdateIndexes
|
|
├─ getOrCreateAddressableNote(address) / getOrCreateNote(id)
|
|
├─ justConsumeInnerInner(event, …) ← per-class `when`: a typed consume(...) overload,
|
|
│ consumeBaseReplaceable, … ; unlisted class → rejected
|
|
└─ if new: hint / mint / gatherer indexes
|
|
consume(...) → refreshNewNoteObservers(note): matching `observables`, then live.newNote(note)
|
|
```
|
|
|
|
Locally signed events enter through `justConsumeMyOwnEvent(event)`; embedded events through
|
|
`consumeEmbedded(event)`.
|
|
|
|
## Lookups
|
|
|
|
| Need | Call |
|
|
|---|---|
|
|
| note by id or address string | `getOrCreateNote(hex)`, `getNoteIfExists(hex)`, `checkGetOrCreateNote(hex)` (null for bad input) |
|
|
| addressable | `getOrCreateAddressableNote(address)`, `getAddressableNoteIfExists(address)` |
|
|
| user | `getOrCreateUser(pubkey)`, `getUserIfExists(pubkey)`, `checkGetOrCreateUser(key)` |
|
|
| live filter results | `observeNotes(filter: Filter): Flow<List<Note>>` |
|
|
| text search | `search` (`CacheSearch.kt`) |
|
|
|
|
`getOrCreate*` never does network I/O and returns the same wrapper for the same id while it is alive.
|
|
|
|
## Reactive consumption
|
|
|
|
- One note: `note.flow()` returns its `NoteFlowSet` (`metadata`, `replies`, `reactions`, `boosts`,
|
|
`zaps`, `reports`, `relays`, `ots`, `edits`, `labels`). `flowSet` is null until someone calls `flow()`.
|
|
- One user's profile: `user.metadata().flow`.
|
|
- Everything: `LocalCache.live.newEventBundles` / `deletedEventBundles` — `SharedFlow<Set<Note>>`
|
|
batched by `BundledInsert` (1 s windows), `DROP_OLDEST` with a 100-bundle buffer. Feeds consume
|
|
it through `ICacheProvider.getEventStream()`.
|
|
|
|
## Replacement and deletion
|
|
|
|
- Regular events: first copy wins; kind-5 deletions go to `deletionIndex`, and a later copy of a
|
|
deleted event is refused (`deletionIndex.hasBeenDeleted`).
|
|
- Replaceable and addressable events: the newest `created_at` replaces `note.event` in place on the
|
|
same wrapper (addressables are keyed by `Address`). Hold the `Note`, never the `Event`.
|
|
|
|
## Pruning and memory
|
|
|
|
- `CachePruner` (`pruner`): `pruneOldMessages`, `pruneHiddenMessages`, `pruneExpiredEvents`,
|
|
`prunePastVersionsOfReplaceables`, `pruneRepliesAndReactions`, `cleanObservers` (calls
|
|
`clearFlow()` on every note).
|
|
- `MemoryTrimmingService` (`model/cache/MemoryTrimmingService.kt`) drives it; `AppModules.kt` and
|
|
`DesktopAppModules.kt` each create one.
|
|
- No TTL beyond that. A long session accumulates; bound retention in the feed (`limit()`), not by
|
|
hoping the cache drops things.
|
|
|
|
## Gotchas
|
|
|
|
- **A new event class must be routed in `justConsumeInnerInner`.** `EventFactory` parsing it is not
|
|
enough; an unlisted class hits the `else` branch and is rejected.
|
|
- **Observers fire inside consume.** `refreshNewNoteObservers` runs as part of `consume(...)`, so
|
|
state the cache derives from that event (channels and similar) may not be filled in yet when a
|
|
collector wakes. Re-read in the collector.
|
|
- **Full scans are expensive.** `notes.filterIntoSet { … }` over the whole cache is what a feed's
|
|
`feed()` does on refresh; never do it per frame or in composition.
|
|
- **Processes**: `LocalCache` lives only in the Android main process — see AGENTS.md *Android Runtime
|
|
Processes*.
|