didactyl/plans/agent_memory.md

Agent Memory System — Implementation Plan

Overview

Add short-term and long-term memory to the Didactyl agent, stored as Nostr kind 30078 (addressable app data) events. Both memory types use freeform markdown content, NIP-44 encrypted (the agent encrypts to itself for privacy).

• Short-term memory — injected into every LLM prompt automatically; the agent's scratchpad for facts, context, and notes that should persist across conversations
• Long-term memory — retrieved on demand via a tool call; a larger store for reference material the agent doesn't need in every prompt

Both are global to the agent (not per-user), stored as single replaceable events keyed by d tag. Content is NIP-44 encrypted to the agent's own public key, ensuring only the agent can read its own memories.

Nostr Event Structure

Short-Term Memory Event

{
  "kind": 30078,
  "content": "<NIP-44 encrypted markdown>",
  "tags": [
    ["d", "short_term_memory"],
    ["app", "didactyl"]
  ]
}

Decrypted content example:

## Current Context

- Working on project Alpha...
- Admin prefers concise responses...

Long-Term Memory Event

{
  "kind": 30078,
  "content": "<NIP-44 encrypted markdown>",
  "tags": [
    ["d", "long_term_memory"],
    ["app", "didactyl"]
  ]
}

Decrypted content example:

## Project Notes

### Alpha
- Started 2026-01-15...

## Preferences
- Admin timezone: UTC-3...

Encryption Details

Memory content is NIP-44 encrypted using the agent's own keypair (self-encryption):

• Encrypt: nostr_nip44_encrypt(agent_private_key, agent_public_key, plaintext, ciphertext, size)
• Decrypt: nostr_nip44_decrypt(agent_private_key, agent_public_key, ciphertext, plaintext, size)

This follows the existing pattern in tool_nostr_dm.c (execute_nostr_encrypt/execute_nostr_decrypt). The d tags remain unencrypted (they must be for addressable event replacement to work), but the actual memory content is private.

Size Limits

Memory Type	Max Content Size	Rationale
Short-term	~8,000 chars	In every prompt; ~2K tokens budget
Long-term	~32,000 chars	On-demand only; well within 64KB relay limit

Architecture

flowchart TD
    A[Agent Startup] --> B[Fetch kind 30078 from nostr]
    B --> B2[NIP-44 decrypt content with agent keys]
    B2 --> C[Cache short_term_memory plaintext in-memory]
    B2 --> D[Cache long_term_memory plaintext in-memory]

    E[Every Prompt Build] --> F[Soul template section: short_term_memory]
    F --> G[tool: memory_short_term_read]
    G --> H[Return cached plaintext STM content]
    H --> I[Injected as system message in prompt]

    J[Agent wants to recall LTM] --> K[Calls my_memory tool]
    K --> L{Cached?}
    L -->|Yes| M[Return cached plaintext LTM]
    L -->|No| N[Query nostr for kind 30078 d=long_term_memory]
    N --> N2[NIP-44 decrypt with agent keys]
    N2 --> O[Cache plaintext result]
    O --> M

    P[Agent wants to save memory] --> Q[Calls memory_save tool]
    Q --> R{type param}
    R -->|short_term| S[NIP-44 encrypt + publish kind 30078]
    S --> S2[Update STM cache with plaintext]
    R -->|long_term| T[NIP-44 encrypt + publish kind 30078]
    T --> T2[Update LTM cache with plaintext]

New Tools

1. memory_save — Write memory

Description: Save content to agent short-term or long-term memory. Publishes as kind 30078 to nostr and updates the local cache.

Parameters:

Param	Type	Required	Description
type	string	yes	"short_term" or "long_term"
content	string	yes	Markdown content to store

Behavior:

1. Validate type is "short_term" or "long_term"
2. Enforce size limit based on type (on the plaintext, before encryption)
3. Determine d-tag: "short_term_memory" or "long_term_memory"
4. NIP-44 encrypt the content (agent encrypts to own public key)
5. Publish kind 30078 event with encrypted content, ["d", <d_tag>] and ["app", "didactyl"] tags via nostr_handler_publish_kind_event()
6. Update the in-memory cache (stores plaintext for fast access)
7. Return success with event_id

2. my_memory — Read long-term memory

Description: Retrieve the agent's long-term memory. Returns cached content or fetches from nostr if not yet loaded.

Parameters:

Param	Type	Required	Description
(none)	—	—	No parameters needed

Behavior:

1. Check if LTM is cached in-memory (plaintext)
2. If cached, return the content
3. If not cached, query nostr for kind 30078 with d=long_term_memory authored by self
4. NIP-44 decrypt the content (agent decrypts with own keypair)
5. Cache the plaintext result
6. Return the content (or empty string if no memory exists yet)

3. memory_short_term_read — Internal tool for prompt injection

Description: Internal tool (not exposed to LLM as a callable tool) used by the soul template to inject short-term memory into every prompt.

Behavior:

1. Return cached short-term memory content
2. If cache is empty, return empty string (skip_if_empty will omit the section)

Files to Modify/Create

New File: src/tools/tool_memory.c

Contains implementations for:

• execute_memory_save() — handles both STM and LTM writes (NIP-44 encrypts before publishing)
• execute_my_memory() — handles LTM reads (NIP-44 decrypts on cache miss)
• execute_memory_short_term_read() — internal tool for prompt template injection (returns cached plaintext)
• Static cache variables for STM and LTM plaintext content with mutex protection
• memory_init() — called at startup to fetch existing memories from nostr and NIP-44 decrypt them into cache
• memory_cleanup() — free cached memory on shutdown
• Helper functions for NIP-44 self-encrypt/decrypt using agent's own keypair (pattern from tool_nostr_dm.c)

Modified: src/tools/tools_internal.h

Add function declarations:

• char* execute_memory_save(tools_context_t* ctx, const char* args_json);
• char* execute_my_memory(tools_context_t* ctx, const char* args_json);
• char* execute_memory_short_term_read(tools_context_t* ctx, const char* args_json);
• int memory_init(tools_context_t* ctx); — fetches from nostr, NIP-44 decrypts, caches plaintext
• void memory_cleanup(void);

Modified: src/tools/tools_dispatch.c

Add dispatch entries:

• "memory_save" → execute_memory_save()
• "my_memory" → execute_my_memory()
• "memory_short_term_read" → execute_memory_short_term_read()

Modified: src/tools/tools_schema.c

Add OpenAI function schemas for:

• memory_save — exposed to LLM (type + content params)
• my_memory — exposed to LLM (no params)
• memory_short_term_read is NOT added to schema (internal only, called by template)

Modified: src/agent.c

• Call memory_init() during agent_init() to load existing memories from nostr on startup
• Call memory_cleanup() during agent_cleanup()
• Add "short_term_memory" to context section detection in detect_context_section()

Modified: Soul template in config.jsonc.example

Add a new template section for short-term memory injection:

- section: short_term_memory
  role: system
  tool: memory_short_term_read
  skip_if_empty: true

This goes after the existing context sections (admin identity, profile, etc.) and before the DM history expand section.

Modified: Makefile

Add src/tools/tool_memory.c to the build.

Startup Flow

1. agent_init() calls memory_init()
2. memory_init() queries nostr for kind 30078 events authored by self with d-tags short_term_memory and long_term_memory
3. For each event found, NIP-44 decrypt the content using the agent's own keypair
4. Decrypted plaintext results are cached in static variables protected by mutex
5. If no events found, caches remain empty (agent starts with blank memory)
6. If decryption fails (e.g., key mismatch from a previous agent identity), log a warning and start with blank memory

Prompt Injection Flow (Short-Term Memory)

1. prompt_template_build_messages() encounters the short_term_memory section
2. Section has tool: memory_short_term_read — calls execute_memory_short_term_read()
3. Tool returns cached STM content as the content field
4. If empty and skip_if_empty: true, section is omitted from prompt
5. If non-empty, injected as a system message like:

   ## Short-Term Memory
   
   <agent's markdown notes here>
   

Cache Design

// In tool_memory.c
static char* g_short_term_memory = NULL;   // cached STM content
static char* g_long_term_memory = NULL;    // cached LTM content  
static int g_stm_loaded = 0;              // whether STM has been fetched
static int g_ltm_loaded = 0;              // whether LTM has been fetched
static pthread_mutex_t g_memory_mutex = PTHREAD_MUTEX_INITIALIZER;

#define MEMORY_STM_MAX_CHARS 8000
#define MEMORY_LTM_MAX_CHARS 32000

Implementation Order

1. Create src/tools/tool_memory.c with cache, init, cleanup, and all three tool functions
2. Add declarations to tools_internal.h
3. Add dispatch entries to tools_dispatch.c
4. Add schemas for memory_save and my_memory to tools_schema.c
5. Wire up memory_init()/memory_cleanup() in agent.c
6. Add short_term_memory template section to soul in config.jsonc.example
7. Update Makefile build
8. Update soul template in system prompt to mention memory capabilities
9. Test: save STM → verify prompt injection; save LTM → verify recall via my_memory