# Context Optimization Plan

Analysis of [`context.log.md`](../context.log.md) (13,609 bytes / ~3,402 tokens across 20 sections) and [`context_template.md`](../context_template.md).

## Issues Found

### 1. Massive Duplication in `startup_events` Section

The **startup_events** section (line 74-80 in the log) dumps the *entire* `config.startup_events` array as raw JSON — including the full soul/system prompt (kind 31120) which is already sent verbatim as the **system_prompt** section. The soul text appears **twice** in every request.

**Estimated waste:** ~1,500-2,000 tokens per request.

**Fix:** Filter out kind 31120 (soul) from the startup_events JSON blob, or better yet, only include kinds the model actually needs to reference (kind 0 profile, kind 10002 relay list, kind 3 contacts). The soul is already the system prompt — repeating it as data is pure waste.

### 2. Duplicate Startup Messages in DM History

The DM history contains **four separate** `Didactyl has started up and is online (version v0.0.29, connected relays: 4/4).` assistant messages (lines 123-127, 130-134, 144-148, 222-226, 245-249). These are startup announcement DMs that got stored as separate events. The model sees the same boilerplate startup message repeated across the conversation.

**Estimated waste:** ~200-300 tokens.

**Fix:** Deduplicate consecutive identical assistant messages in the DM history builder, or filter out startup announcement messages (they carry no conversational value).

### 3. Skills Rendered as Raw JSON Instead of Structured Text

Skill instructions at lines 97-118 are dumped as raw JSON objects. Models parse structured natural language far more reliably than nested JSON. The `content_fields` serialization format wastes tokens on JSON syntax characters and key quoting.

**Estimated waste:** ~100-200 tokens of JSON overhead per skill, plus reduced comprehension quality.

**Fix:** When serializing `content_fields`-based skills for context, flatten them into readable text:
```
Skill: long_form_note
Description: How to publish a NIP-23 long-form article (kind 30023)
NIP: NIP-23
Event Kind: 30023
Format: The content field must be markdown text...
Required Tags:
  - d: Addressable identifier slug...
  - title: Human-readable article title
  - published_at: Unix timestamp as string...
Procedure:
  1. Determine title and d tag...
  2. Draft markdown body content...
```

### 4. Empty Sections Still Sent

The **admin_relay_list** section (line 65-71) has no data — the JSON value is empty. Sending an empty section wastes tokens on the header/framing with no informational value.

**Estimated waste:** ~30-40 tokens.

**Fix:** Skip sections where the resolved variable is empty or whitespace-only.

### 5. Admin Identity Could Be Merged with Admin Profile

The **admin_identity** section (line 47-53) sends just the hex pubkey, then **admin_profile** (line 56-62) sends the full kind 0 JSON which implicitly identifies the admin. These could be a single section.

**Estimated savings:** ~40-50 tokens of framing overhead.

### 6. `admin_notes` Placement Breaks Conversation Flow

In the template, `admin_notes` is placed *after* `dm_history` (expand). In the actual log, this means a system message appears sandwiched between DM history messages (line 252, between assistant messages and the final user message at line 274). This breaks the natural conversation flow and may confuse the model about message ordering.

**Fix:** Move `admin_notes` *before* `dm_history` in the template so all system context is grouped together before the conversation begins.

### 7. No Agent Self-Identity Section

The model knows it is Didactyl from the system prompt, but there is no section telling it its own pubkey/npub. The admin pubkey is provided but the agent's own key is not in the context (it is only available via the `nostr_pubkey` tool). Adding a small self-identity section would let the model reference its own key without a tool call.

**Estimated cost:** ~20-30 tokens.

## Priority Summary

| Priority | Issue | Token Savings | Complexity |
|----------|-------|---------------|------------|
| **P0** | Soul duplicated in startup_events | ~1,500-2,000 | Low — filter kind 31120 from startup blob |
| **P1** | Duplicate startup DMs in history | ~200-300 | Medium — dedup logic in history builder |
| **P1** | Skills as raw JSON | ~100-200 + quality | Medium — flatten content_fields to text |
| **P2** | Empty sections still sent | ~30-40 | Low — skip empty resolved vars |
| **P2** | admin_notes after dm_history | 0 (quality) | Low — reorder template |
| **P3** | Merge admin_identity + admin_profile | ~40-50 | Low — template change |
| **P3** | Add agent self-identity section | -20-30 (adds) | Low — new template var |

## Bug: Kind 10002 Relay List Is Always Empty

At [`nostr_handler.c:705`](../src/nostr_handler.c:705) the kind 10002 handler stores `content->valuestring`, but NIP-65 relay list events have an **empty content field** — the relay URLs live in the **tags** as `["r", "wss://relay.example.com"]` entries. So `g_admin_kind10002_json` is always `""`.

**Fix:** Parse the `"r"` tags from the kind 10002 event and serialize them as a JSON array of relay URL strings (or plain-text list).

## Sender Verification Status

The [`tier`](../src/nostr_handler.h:8) enum (`DIDACTYL_SENDER_ADMIN`, `DIDACTYL_SENDER_WOT`, `DIDACTYL_SENDER_STRANGER`) is already resolved before [`agent_on_message()`](../src/agent.c:1453) is called, but it is **not passed into the context builder**. The model has no way to know whether the current message was cryptographically verified as coming from the administrator vs. a web-of-trust contact.

**Fix:** Pass the sender tier into the context builder and expose it as a template variable (e.g. `{{sender_verification}}`) that resolves to text like:
- `"This message has been cryptographically verified as coming from your administrator."`
- `"This message is from a web-of-trust contact (not the administrator)."`

## Proposed Optimized Template

```yaml
- section: agent_identity
  role: system
  content: |
    Agent Identity
    Your pubkey (hex): {{agent_pubkey}}

- section: sender_context
  role: system
  content: |
    {{sender_verification}}

- section: admin_context
  role: system
  content: |
    Administrator Context

    Pubkey (hex): {{admin_pubkey}}
    {{admin_profile_plain}}
    {{admin_relay_list_plain}}

- section: startup_events
  role: system
  skip_if_empty: true
  content: |
    Startup Events Memory
    {{startup_events_json}}

- section: adopted_skills
  role: system
  skip_if_empty: true
  content: |
    {{adopted_skills_content}}

- section: admin_notes
  role: system
  skip_if_empty: true
  content: |
    Administrator Recent Notes (source: nostr kind 1)
    {{admin_notes_content}}

- section: dm_history
  role: expand
  limit: 12
```

Key changes from current template:
- **No markdown headers** in system sections — plain English throughout
- **Merged admin section** combines identity, profile, and relay list
- **`{{admin_profile_plain}}`** — new variable that renders kind 0 JSON as readable text (e.g. `Name: WSB, About: ...`)
- **`{{admin_relay_list_plain}}`** — new variable that renders relay URLs from tags as a plain list
- **`{{sender_verification}}`** — new variable stating cryptographic verification status
- **`admin_notes` moved before `dm_history`** so all system context is grouped before conversation
- **`skip_if_empty`** prevents sending empty sections

## Implementation Steps

1. **Fix kind 10002 relay list bug** — extract relay URLs from tags instead of content in [`nostr_handler.c:705`](../src/nostr_handler.c:705)
2. **Filter kind 31120** from `startup_events_json` variable resolver in [`agent.c`](../src/agent.c)
3. **Deduplicate consecutive identical messages** in DM history builder
4. **Flatten `content_fields` JSON skills** into readable text format
5. **Add `skip_if_empty` support** to template engine (skip section when resolved content is blank)
6. **Reorder template** — move `admin_notes` before `dm_history`
7. **Add `agent_pubkey` template variable** and agent identity section
8. **Merge admin sections** — combine identity + profile (plain English) + relay list into one section
9. **Add `admin_profile_plain` variable** — parse kind 0 JSON into readable text
10. **Add `admin_relay_list_plain` variable** — parse kind 10002 tags into relay URL list
11. **Pass sender tier to context builder** and add `sender_verification` template variable
12. **Remove markdown formatting** from system section content — use plain English