didactyl/plans/tool_orchestration.md

Tool Orchestration Plan

Overview

This plan covers four related features:

1. Slash commands — direct tool execution bypassing the LLM
2. Skill-forward execution (/run + skill_run) — one-shot skill invocation with LLM, including external skill sharing
3. Skill-tool maturity levels — skills that register as callable tools with promotion workflow
4. Deterministic step executor — hardened skills that run without LLM involvement

1. Slash Commands (Direct Tool Execution)

Behavior

When a message starts with /, bypass the LLM entirely and execute the tool directly.

/local_shell_exec {"command": "ls -la"}
/nostr_query {"filter": {"kinds": [1], "limit": 5}}
/nostr_nip05_lookup {"identifier": "jack@cash.app"}

Parsing Rules

• /tool_name — call tool with empty args {}
• /tool_name {"key": "value"} — call tool with JSON args
• /tool_name plain text — wrap as {"input": "plain text"} (convenience)
• /help — list available tools
• /help tool_name — show tool schema

Implementation

1. In agent_on_message(), check if message[0] == '/'
2. Parse tool name (everything between / and first space or end)
3. Parse args (everything after tool name, try JSON first, fall back to string wrapper)
4. Call tools_execute() directly
5. Send result as DM — no LLM round-trip
6. Still log to context.log.md with phase=direct_tool_exec

Special Slash Commands

• /help — list available tools
• /help tool_name — show tool schema
• /run — skill-forward execution (see section 2)

Security

• Only admin tier can use slash commands (same as current tool policy)
• Slash commands respect the same tools.enabled and security.admin.tools_enabled config flags

2. Skill-Forward Execution

The Problem

Today, skills are passive — they're injected into the LLM context via append_adopted_skills_context() and the LLM decides when they're relevant. There's no way to say "run this specific skill right now" and there's no way to try someone else's skill without permanently adopting it.

Three Invocation Layers

Layer	How it works	LLM?	Persists?
Passive/adopted	Skill instructions injected into every conversation context	Yes, LLM decides relevance	Yes — in kind 10123 adoption list
Skill-forward	Skill instructions become the primary system prompt; LLM executes them	Yes, but constrained	No — one-shot execution
Hardened	Deterministic step executor, no LLM	No	Yes — adopted skill with execution: hardened

Entry Points

A. /run slash command (admin direct invocation)

/run deploy-website staging                         # run own adopted skill by d_tag
/run 31123:<pubkey>:deploy-website staging           # run anyone's skill by address
/run deploy-website {"target": "production"}         # JSON args

Parsing:

1. First token after /run is the skill identifier (d_tag or kind:pubkey:d_tag address)
2. Everything after is args (try JSON first, fall back to {"input": "plain text"})

B. skill_run tool (LLM-mediated invocation)

The LLM can invoke skills on behalf of the admin during conversation:

{
  "name": "skill_run",
  "description": "Fetch and execute a skill one-shot without adopting it. Works with own adopted skills by d_tag or any public skill by address.",
  "parameters": {
    "type": "object",
    "properties": {
      "d_tag":    { "type": "string", "description": "Skill d_tag for own adopted skills" },
      "address": { "type": "string", "description": "Full skill address: kind:pubkey:d_tag" },
      "pubkey":  { "type": "string", "description": "Author pubkey, used with d_tag to form address" },
      "args":    { "type": "string", "description": "Arguments or context to pass to the skill" },
      "sandbox": { "type": "boolean", "description": "Override sandbox setting. Default: true for external, false for own skills" }
    }
  }
}

This enables natural conversation like:

"My friend @jack just published a skill called summarize-thread. Try it on the latest thread in my feed."

The agent would skill_search to find it, then skill_run to execute it.

Execution Flow

Both /run and skill_run use the same underlying executor:

graph TD
    A[/run or skill_run called] --> B{Skill identifier type?}
    B -->|d_tag only| C[Look up in adopted skills cache]
    B -->|address or pubkey+d_tag| D[Fetch skill event from Nostr]
    C --> E{Found?}
    D --> E
    E -->|No| F[Return error: skill not found]
    E -->|Yes| G{External skill?}
    G -->|Yes| H[Apply sandbox - restrict tools]
    G -->|No| I[Full tool access]
    H --> J[Build skill-forward prompt]
    I --> J
    J --> K[System: base context + skill instructions]
    K --> L[User: args as user message]
    L --> M[LLM call with tools]
    M --> N[Return result to caller]

Skill-Forward Prompt Construction

Reuses the pattern from agent_on_trigger():

[base system context / soul]

Skill execution context:
- You are executing a specific skill on demand.
- Follow the skill instructions below precisely.
- The user's arguments provide the context for this execution.
- Keep output concise and actionable.

Skill d_tag: deploy-website
Skill address: 31123:<pubkey>:deploy-website
Skill source: [own | external:<author_display_name>]

Skill instructions:
[skill content here]

User message:

[args provided by caller]

The prompt is so skill-forward that the LLM has no real option but to execute the skill instructions against the provided args.

Sandbox for External Skills

When executing a skill from another author (not own pubkey), a tool sandbox is applied by default:

Allowed tools (safe/read-only):

• nostr_query — read Nostr events
• nostr_nip05_lookup — NIP-05 lookups
• nostr_post — publish events (the agent signs, so this is safe)
• nostr_list_manage — manage lists
• skill_list, skill_search — read skill metadata

Blocked tools (destructive/dangerous):

• local_shell_exec — arbitrary command execution
• local_file_read, local_file_write — filesystem access
• Any future tools marked as destructive: true

Override: The admin can explicitly opt in to full tool access:

• /run --unsafe 31123:<pubkey>:risky-skill args
• skill_run with sandbox: false

Skill Sharing on Nostr

graph TD
    A[Friend creates skill] -->|kind 31123| B[Published on Nostr relays]
    B --> C{How do you find it?}
    C -->|skill_search popular:true| D[Discovery via WoT adoption lists]
    C -->|Friend tells you the d_tag| E[Direct reference]
    C -->|skill_search pubkey:friend| F[Browse friends skills]
    D --> G[skill_run - try it once, sandboxed]
    E --> G
    F --> G
    G -->|Liked it?| H{Adopt?}
    H -->|Yes| I[skill_adopt - permanent]
    H -->|No| J[Done - nothing persisted]
    I --> K[Shows in adopted skills context]
    K --> L[Agent uses it automatically]
    L -->|Or invoke directly| M[/run skill-d_tag args]

3. Skill-Tool Maturity Levels

Skills can declare an execution maturity level that determines how they run:

Level	Execution	LLM?	Use case
draft	LLM interprets procedure text from skill instructions	Yes	Exploring and iterating on a workflow
guided	LLM with forced tool_choice + parameter defaults	Yes, constrained	Workflow is stable but needs LLM judgment
hardened	Deterministic step executor, no LLM	No	Workflow is proven and should run exactly as defined

Skill Definition Extensions

kind: 31123
d: deploy_website
execution: hardened
tool_schema:
  name: deploy_website
  description: Build and deploy the static website
  parameters:
    target:
      type: string
      enum: [staging, production]
      default: staging
steps:
  - tool: local_shell_exec
    args:
      command: "make build TARGET={{target}}"
  - tool: local_shell_exec
    args:
      command: "rsync -av dist/ server:/var/www/{{target}}/"
  - return: "Deployed to {{target}}"

How Each Level Works

draft: Current behavior. Skill instructions are injected into context. LLM reads them and decides which tools to call. No special handling needed.

guided: Agent sets tool_choice to the skill's preferred tool. Parameter defaults from the skill are merged with the model's generated arguments (skill defaults win on conflict). Reduces LLM freedom while still allowing it to fill in dynamic values.

hardened: Agent executes the steps array directly using the deterministic step executor. No LLM call at all. The skill becomes equivalent to a slash command.

Promotion Workflow

1. Admin iterates with LLM on a task (draft)
2. Admin saves working procedure as a skill: skill_create with execution: draft
3. Admin tests, refines, promotes: update skill to execution: guided
4. Once proven reliable, promote to execution: hardened with explicit steps
5. Hardened skills become available as slash commands: /deploy_website {"target": "production"}

4. Deterministic Step Executor

A simple sequential executor for hardened skills.

Step Types

steps:
  # Execute a tool
  - tool: local_shell_exec
    args: {command: "ls -la"}
    save_as: listing          # optional: save result to variable

  # Execute a tool with variable substitution
  - tool: nostr_post
    args:
      kind: 30023
      content: "{{file_content}}"
      tags: [["d", "{{d_tag}}"]]

  # Conditional - simple
  - if: "{{listing.success}}"
    then:
      - tool: local_shell_exec
        args: {command: "echo done"}
    else:
      - return: "Failed: {{listing.error}}"

  # Return final result
  - return: "Published to {{d_tag}}"

Variable Substitution

• {{param_name}} — from tool parameters provided by caller
• {{step_name.field}} — from a previous step's result (requires save_as)
• Simple string replacement, no expression evaluation

Implementation in C

• Parse steps array from skill content (JSON or YAML)
• Iterate steps sequentially
• For each tool step: call tools_execute(), optionally save result
• For each return step: substitute variables and return string
• For each if step: evaluate truthiness of variable, branch accordingly
• Total implementation: ~200-400 lines of C

Error Handling

• If any tool step fails (returns success: false), abort and return the error
• Optional on_error field per step for custom error messages
• Timeout inherited from tool config

5. Tool Registration for Skill-Tools

Hardened and guided skills with a tool_schema field get registered in the tools array at runtime.

At Skill Refresh Time

1. Parse tool_schema from skill content
2. Generate OpenAI function schema from it
3. Append to the tools array returned by tools_build_openai_schema_json()
4. When model calls the skill-tool, route to skill executor instead of hardcoded C function

In tools_execute()

1. Check if tool_name matches a hardcoded tool — execute normally
2. If not, check if it matches a registered skill-tool
3. If guided: run sub-LLM call with skill procedure + forced tool_choice
4. If hardened: run deterministic step executor

6. Security Model

Tool Classification

Tools are classified for sandbox purposes:

typedef enum {
    TOOL_SAFETY_SAFE,        // read-only or agent-signed actions
    TOOL_SAFETY_DESTRUCTIVE  // filesystem, shell, or external system mutations
} tool_safety_t;

Tool	Safety	Reason
nostr_query	safe	Read-only
nostr_nip05_lookup	safe	Read-only
nostr_post	safe	Agent signs, admin controls keys
nostr_list_manage	safe	Agent signs
skill_list	safe	Read-only
skill_search	safe	Read-only
skill_run	safe	Recursive execution uses its own sandbox
local_shell_exec	destructive	Arbitrary command execution
local_file_read	destructive	Filesystem access
local_file_write	destructive	Filesystem mutation

Sandbox Rules

Scenario	Default sandbox	Override
Own adopted skill via /run d_tag	No sandbox	N/A
External skill via /run address	Sandbox ON	/run --unsafe address
skill_run tool, own skill	No sandbox	sandbox: true
skill_run tool, external skill	Sandbox ON	sandbox: false
Hardened skill steps	No sandbox (steps are explicit)	N/A

Future Considerations

• Skill sharing: Hardened skill-tools could be shared between agents via Nostr (kind 31123 events). Another agent adopts the skill and gets the tool automatically.
• Versioning: Skills already use addressable events (d-tag). Updating a skill automatically updates the tool.
• Permissions: Skill-tools could have their own permission model (e.g., some skill-tools available to WoT contacts).
• Composability: Skill-tools calling other skill-tools (nested execution with sandbox inheritance).
• Dry-run mode: A future /run --dry flag that shows what tools would be called without executing them.
• Skill ratings: Agents could publish ratings/reviews of skills they've tried, building a WoT-based skill marketplace.

Implementation Priority

1. Slash commands (direct tool execution) — simplest, highest immediate value
2. /run for own adopted skills — skill-forward execution of already-adopted skills
3. skill_run tool + external skill fetching — enables "try my friend's skill" flow
4. External skill sandbox — tool safety classification and sandbox enforcement
5. Hardened skill-tool step executor — enables deterministic workflows
6. Skill-tool registration in tools array — makes skill-tools visible to LLM
7. Guided execution with forced tool_choice — bridges draft and hardened
8. Promotion workflow UX — admin commands to change skill maturity level