mirror of
https://github.com/vitorpamplona/amethyst.git
synced 2026-10-30 15:48:36 +00:00
Quartz moves to its own repo, the way the crypto primitives moved to crypto-kmp. Every module depends on com.vitorpamplona.quartz:quartz 2.0.0 (libs.quartz) instead of :quartz, and -PquartzDir=../quartz builds a local checkout instead of the artifact, for every module and for quic's own quartz dependency (which replaces the root build's substitution of the published quartz with :quartz). What moved with quartz: its CI jobs (lint tasks, test-quartz-linux-native, test-quartz-ios), the Maven Central publish step of the release workflow, jitpack.yml, tools/unicode-nfkc, the nostr-expert and event-store-semantics skills, the agent-skill check, and the README's library usage docs. What stays here: - The four quartz client tests that need a real Ktor socket relay (TrustNetworkSync*, GiantReqStreamTest, BulkDownloadBenchmark) move to geode's tests; geode forwards -PprodRelayBench for the latter. quartz's other relay-client tests now use a test relay of its own, which ends quartz's test dependency on geode. - The benchmark module's resources (previously read from quartz's androidDeviceTest resources) move into benchmark/; relayBench --real reads the corpus from there. - NoteComposeKindCoverageTest reads EventFactory.kt from quartz's JVM sources jar, or from the -PquartzDir checkout, passed by commonsUI's build. - commons and commonsUI declare crypto-kmp (and commons mls-kmp), which they import directly. Through quartz's api alone, a -PcryptoKmpDir checkout never reached the common metadata compile. - The catalog drops what only quartz used (kchesslib, negentropy, jackson-databind, sqlite-bundled, secp256k1-kmp common, compose runtime-annotation, the maven-publish plugin). agent_config_check checks a skill's quartz/... paths against a ../quartz checkout when there is one and skips them otherwise. AGENTS.md, the skills, CONTRIBUTING, BUILDING, RELEASE_OPS, PLANS and the READMEs point at the new repo. Needs quartz 2.0.0 (and crypto-kmp 1.0.0) on Maven Central before CI can resolve it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DEkJaDWYrcTbGvT8JLQCrG
5.0 KiB
5.0 KiB
LocalCache: the in-memory event store
commons/src/commonMain/kotlin/com/vitorpamplona/amethyst/commons/model/cache/:
LocalCache.kt—object LocalCache : EventCache(), the process-wide instance both app modules use.EventCache.kt— all the logic (open class EventCache : ILocalCache, ICacheProvider, Dao). It is a class so a test can build a cache of its own instead of sharing the singleton.
Structures (fields of EventCache)
val users = LargeSoftCache<HexKey, User>()
val notes = LargeSoftCache<HexKey, Note>()
val addressables = LargeSoftCache<Address, AddressableNote>()
val chatroomList = LargeCache<HexKey, ChatroomList>()
val publicChatChannels = LargeCache<HexKey, PublicChatChannel>()
val liveChatChannels = LargeCache<Address, LiveActivitiesChannel>()
// … ephemeralChannels, geohashChannels, relayGroupChannels, concordChannels
val deletionIndex = DeletionIndex()
val observables = FilterIndex<Observable>()
val pruner = CachePruner(this)
val search = CacheSearch(this)
val live: LocalCacheFlow = LocalCacheFlow()
LargeSoftCache (LargeSoftCache.kt) holds weak references (a ConcurrentSkipListMap of weak
refs on JVM/Android). A Note or User nobody holds strongly can be collected, which is why every
account state object keeps its note in a field. Channel maps are plain LargeCache (quartz utils/cache/LargeCache.kt; see
quartz's nostr-expert skill).
Insertion path
relay EVENT
└─ CacheClientConnector (commons/src/jvmAndroid/.../relayClient/) → cache.justConsume(event, relay, false)
├─ already deleted? → re-send the deletion to that relay, drop the event
├─ addressable and we hold a newer version? → re-send ours to that relay
└─ justConsumeAndUpdateIndexes
├─ getOrCreateAddressableNote(address) / getOrCreateNote(id)
├─ justConsumeInnerInner(event, …) ← per-class `when`: a typed consume(...) overload,
│ consumeBaseReplaceable, … ; unlisted class → rejected
└─ if new: hint / mint / gatherer indexes
consume(...) → refreshNewNoteObservers(note): matching `observables`, then live.newNote(note)
Locally signed events enter through justConsumeMyOwnEvent(event); embedded events through
consumeEmbedded(event).
Lookups
| Need | Call |
|---|---|
| note by id or address string | getOrCreateNote(hex), getNoteIfExists(hex), checkGetOrCreateNote(hex) (null for bad input) |
| addressable | getOrCreateAddressableNote(address), getAddressableNoteIfExists(address) |
| user | getOrCreateUser(pubkey), getUserIfExists(pubkey), checkGetOrCreateUser(key) |
| live filter results | observeNotes(filter: Filter): Flow<List<Note>> |
| text search | search (CacheSearch.kt) |
getOrCreate* never does network I/O and returns the same wrapper for the same id while it is alive.
Reactive consumption
- One note:
note.flow()returns itsNoteFlowSet(metadata,replies,reactions,boosts,zaps,reports,relays,ots,edits,labels).flowSetis null until someone callsflow(). - One user's profile:
user.metadata().flow. - Everything:
LocalCache.live.newEventBundles/deletedEventBundles—SharedFlow<Set<Note>>batched byBundledInsert(1 s windows),DROP_OLDESTwith a 100-bundle buffer. Feeds consume it throughICacheProvider.getEventStream().
Replacement and deletion
- Regular events: first copy wins; kind-5 deletions go to
deletionIndex, and a later copy of a deleted event is refused (deletionIndex.hasBeenDeleted). - Replaceable and addressable events: the newest
created_atreplacesnote.eventin place on the same wrapper (addressables are keyed byAddress). Hold theNote, never theEvent.
Pruning and memory
CachePruner(pruner):pruneOldMessages,pruneHiddenMessages,pruneExpiredEvents,prunePastVersionsOfReplaceables,pruneRepliesAndReactions,cleanObservers(callsclearFlow()on every note).MemoryTrimmingService(model/cache/MemoryTrimmingService.kt) drives it;AppModules.ktandDesktopAppModules.kteach create one.- No TTL beyond that. A long session accumulates; bound retention in the feed (
limit()), not by hoping the cache drops things.
Gotchas
- A new event class must be routed in
justConsumeInnerInner.EventFactoryparsing it is not enough; an unlisted class hits theelsebranch and is rejected. - Observers fire inside consume.
refreshNewNoteObserversruns as part ofconsume(...), so state the cache derives from that event (channels and similar) may not be filled in yet when a collector wakes. Re-read in the collector. - Full scans are expensive.
notes.filterIntoSet { … }over the whole cache is what a feed'sfeed()does on refresh; never do it per frame or in composition. - Processes:
LocalCachelives only in the Android main process — see AGENTS.md Android Runtime Processes.