didactyl/plans/template_tool_unification.md

Plan: Unify Template Variables with Tools

Problem

The soul template system (---template--- in kind 31120) uses {{variable}} placeholders that are resolved by a hardcoded C function (agent_template_resolve_var()). This is a parallel data-fetching system alongside the existing tool registry. Every new context data source requires:

1. Adding a C function to produce the data
2. Adding an if (strcmp(...)) branch in the resolver
3. Documenting the new variable name
4. Hoping the template author uses the exact right name

This caused the bug that started this investigation — 5 out of 9 template variables were misspelled/mismatched, producing empty context sections and a provider rejection.

Goal

Eliminate template variables entirely. Template sections fetch their data by calling tools. The tool registry is the single mechanism for getting data into context.

Design

New Template Syntax

Replace content: | with tool: and optional args: directives:

---template---

- section: admin_identity
  role: system
  tool: admin_identity
  skip_if_empty: true

- section: admin_profile
  role: system
  tool: nostr_admin_profile
  skip_if_empty: true

- section: admin_contacts
  role: system
  tool: nostr_admin_contacts
  skip_if_empty: true

- section: admin_relays
  role: system
  tool: nostr_admin_relays
  skip_if_empty: true

- section: admin_notes
  role: system
  tool: nostr_admin_notes
  skip_if_empty: true

- section: tools
  role: system
  tool: tool_list
  skip_if_empty: true

- section: tasks
  role: system
  tool: task_list
  skip_if_empty: true

- section: dm_history
  role: expand
  limit: 12

- section: conversation
  role: user
  tool: message_current
  skip_if_empty: true

A section can still use the old content: with {{variables}} for static text or mixed content. But the primary mechanism for dynamic data is tool:.

How It Works

flowchart TD
    DM[Incoming DM] --> BUILD[Build context from template]
    BUILD --> SOUL[Emit personality as system msg]
    SOUL --> LOOP[For each template section]
    LOOP --> CHECK{Has tool: directive?}
    CHECK -->|Yes| EXEC["tools_execute(tool_name, args)"]
    EXEC --> FORMAT[Extract content from JSON result]
    FORMAT --> EMIT[Emit as chat message]
    CHECK -->|No| RESOLVE["Resolve {{vars}} from content template<br/>(legacy path, eventually removed)"]
    RESOLVE --> EMIT
    EMIT --> LOOP
    LOOP --> DONE[Complete messages array]

New Context Tools

Add lightweight "context tools" that read from cached in-memory data. These are fast (no network), deterministic, and use the same tools_execute() interface as everything else.

Tool	Returns	Source
admin_identity	Admin pubkey + verification text	config + sender tier
nostr_admin_profile	Admin kind 0 profile JSON	cached g_admin_kind0_json
nostr_admin_contacts	Admin kind 3 contacts JSON array	cached g_admin_wot_contacts
nostr_admin_relays	Admin kind 10002 relay list JSON	cached g_admin_kind10002_json
nostr_admin_notes	Admin recent kind 1 notes	cached g_admin_kind1_notes
task_list	Current task list from tasks.json	file read (already exists as task_manage with action: list)
message_current	Current user message text	passed via tool context
agent_identity	Agent pubkey, npub	config

Existing tools that already work for context:

• tool_list — returns tool schemas (already exists)
• task_manage with {"action":"list"} — returns tasks (already exists)
• local_file_read — can read any file (already exists)

Tool Result → Message Content

Tool results are JSON objects like {"success":true,"content":"..."}. The template system extracts the content field (or a configurable field) as the message text. If the tool returns an error or empty content, and skip_if_empty: true is set, the section is omitted.

For tools that return structured data (like tool_list returning a schema array), the template can specify a result_field or format to control extraction:

- section: tools
  role: system
  tool: tool_list
  result_field: tools
  skip_if_empty: true

Implementation Phases

Phase 1: Add context tools + tool: directive support

1. Add new context_* tools to tools.c that wrap the existing cached data getters
2. Extend prompt_template_section_t with tool_name and tool_args fields
3. Extend prompt_template_parse() to recognize tool: and args: directives
4. Extend prompt_template_build_messages() to call tools_execute() when a section has tool_name set
5. Update the soul template in config.jsonc to use tool: directives
6. Keep content: + {{variable}} working as a fallback for backward compatibility

Phase 2: Migrate all sections to tool-based

1. Convert all template sections from content: {{variable}} to tool: directives
2. Verify all context data flows through tools
3. Update context_template.md to reflect new syntax

Phase 3: Remove legacy variable resolver

1. Remove agent_template_resolve_var() and all its helper functions that are now redundant
2. Remove prompt_var_resolver_fn callback from prompt_template_build_messages()
3. Clean up dead code in agent.c
4. Update all documentation

Template Section Struct Changes

typedef struct {
    char name[PROMPT_TEMPLATE_MAX_NAME_LEN];
    char role[PROMPT_TEMPLATE_MAX_ROLE_LEN];
    char* content_template;       // legacy: {{variable}} content
    char* tool_name;              // NEW: tool to call for content
    char* tool_args;              // NEW: JSON args for tool call
    char* result_field;           // NEW: which JSON field to extract (default: "content")
    int limit;
    int skip_if_empty;
    char* provider_name;
    char* provider_content_template;
} prompt_template_section_t;

Soul Template Example (After Migration)

# Didactyl Agent

You are Didactyl, a sovereign AI agent living on Nostr.
...

---template---

- section: admin_identity
  role: system
  tool: admin_identity
  skip_if_empty: true

- section: admin_profile
  role: system
  tool: nostr_admin_profile
  skip_if_empty: true

- section: admin_contacts
  role: system
  tool: nostr_admin_contacts
  skip_if_empty: true

- section: admin_relays
  role: system
  tool: nostr_admin_relays
  skip_if_empty: true

- section: admin_notes
  role: system
  tool: nostr_admin_notes
  skip_if_empty: true

- section: tools
  role: system
  tool: tool_list
  result_field: tools
  skip_if_empty: true

- section: tasks
  role: system
  tool: task_manage
  args: {"action":"list"}
  result_field: tasks
  skip_if_empty: true

- section: dm_history
  role: expand
  limit: 12

- section: conversation
  role: user
  tool: message_current
  skip_if_empty: true

What Gets Deleted Eventually

• agent_template_resolve_var() and all its if (strcmp(...)) branches
• build_tool_schemas_json_string()
• build_tasks_content_string() (replaced by task_manage tool)
• build_admin_recent_posts_text() (replaced by nostr_admin_notes tool)
• build_admin_profile_plain_text() (replaced by nostr_admin_profile tool)
• build_admin_relay_list_plain_text() (replaced by nostr_admin_relays tool)
• build_sender_verification_text() (folded into admin_identity tool)
• build_startup_events_json_string() (replaced by tool if needed)
• build_adopted_skills_payload_string() (replaced by tool if needed)
• nostr_handler_get_admin_kind3_context() (just added, will be replaced by tool)
• The prompt_var_resolver_fn callback type

Benefits

1. Single data-fetching mechanism — tools are the only way to get data
2. No more variable name mismatches — tool names are validated at registration
3. User-configurable context — admin can add any tool output to context by editing the soul template
4. Testable — --test-tool nostr_admin_profile shows exactly what goes into context
5. Self-documenting — tool_list shows all available context tools with descriptions
6. Extensible — new tools automatically become available as context sources
7. Provider overrides still work — provider: directive can still override formatting per-provider

Risks

1. Performance — Tool calls add function dispatch overhead vs direct variable resolution. Mitigated: context tools read cached data, no network calls.
2. Backward compatibility — Old soul templates with {{variables}} would break. Mitigated: Phase 1 keeps both paths working.
3. Tool context threading — tools_execute() needs access to the tools context, which prompt_template_build_messages() doesn't currently have. Solution: pass tools_context_t* to the build function.