Files
didactyl/plans/skill_tool_policy.md

239 lines
10 KiB
Markdown

# Skill Tool Policy — Implementation Plan
> **Goal:** Let a skill define or limit which tools are exposed to the LLM when
> that skill runs. A skill's `tools` tag becomes an enforced allowlist instead
> of an ignored annotation.
## Problem
Skills already declare a `tools` tag, e.g. the swarm worker skill:
```json
["tools", "swarm_read,swarm_claim,swarm_contribute,nostr_post,nostr_query"]
```
But the trigger path ignores it. In [`agent_on_trigger()`](../src/agent.c:1833):
```c
char* tools_json = tools_build_openai_schema_json(&g_tools_ctx);
```
and [`tools_build_openai_schema_json()`](../src/tools/tools_schema.c:2574) returns
the **full 92-tool schema** regardless of the skill. So a worker skill that lists
five tools is actually offered all 92 — including `nostr_post`, which the LLM
used to hand-roll malformed swarm events instead of calling `swarm_claim`.
This is a known gap: [`plans/swarm.md`](swarm.md:256) — *"The stored trigger tool
policy is not enforced on this path."* The docs already promise the behavior:
[`docs/TOOLS.md`](../docs/TOOLS.md:745) and [`docs/CONTEXT.md`](../docs/CONTEXT.md:98).
## What already exists (reuse, do not rebuild)
The plumbing is mostly in place:
| Piece | Location | Status |
|-------|----------|--------|
| `tools_policy[256]` field on the trigger | [`active_trigger_t`](../src/trigger_manager.h:52) | Exists |
| Parse `tools` tag from skill | [`parse_trigger_runtime_tags()`](../src/trigger_manager.c:551) | Exists |
| Store policy on trigger | [`trigger_manager_add()`](../src/trigger_manager.c:1449) | Exists |
| Expose policy in trigger list JSON | [`trigger_manager.c`](../src/trigger_manager.c:1906) | Exists |
| **Apply policy to the schema** | — | **Missing** |
| **Pass policy into `agent_on_trigger()`** | [`execute_llm_action()`](../src/trigger_manager.c:921) | **Missing** |
So the work is: (1) a schema-filtering function, (2) thread the policy through
to the trigger execution, (3) enforce at execution time as defense in depth.
## Design
### Policy format
The `tools` tag is a comma-separated list of tool names:
```
swarm_read,swarm_claim,swarm_contribute,nostr_post,nostr_query
```
**This is a whitelist (allowlist).** The named tools are the *only* tools
exposed. There is no blacklist form in this design.
#### Wildcard
Support an explicit allow-all token so a skill can state its intent rather than
rely on omission:
- `*` (or the word `all`) → expose every tool. Equivalent to omitting the tag.
This matters because "no tag" and "tag = `*`" should mean the same thing, but
the explicit form is self-documenting and lets a skill author say "I deliberately
want everything" instead of leaving it ambiguous.
#### Semantics
| `tools` tag value | Result |
|-------------------|--------|
| absent | all tools (backward compatible) |
| empty string | all tools |
| `*` or `all` | all tools (explicit wildcard) |
| `swarm_read,swarm_claim` | only those two tools |
| `swarm_read, bogus_tool` | only `swarm_read`; `bogus_tool` skipped + warned |
- **Tag present, non-empty, not a wildcard** → allowlist. Only the named tools
are exposed.
- **Tag absent or empty** → all tools exposed (backward compatible; matches
[`docs/TOOLS.md`](../docs/TOOLS.md:745): *"If a skill has no `requires_tool`
tags, all available tools are exposed."*).
- **Unknown tool name in the list** → ignored (do not fail the skill). Log a
warning so typos are visible.
- **Whitespace** around names is trimmed.
#### Why whitelist, not blacklist
- **Safer default.** A new tool added to the codebase is *not* automatically
granted to every skill. With a blacklist, every new tool is silently exposed
everywhere until someone remembers to deny it.
- **Matches the existing docs.** [`docs/TOOLS.md`](../docs/TOOLS.md:745) already
frames this as "only the required and optional tools declared by the skill are
exposed" — a whitelist.
- **Least privilege.** The swarm bug is precisely a case where a worker should
*not* have had `nostr_post`. A whitelist expresses that directly.
A blacklist form (e.g. `-local_shell_exec`) is listed as a possible follow-up,
but is not part of this plan. If both are ever supported, the rule should be:
whitelist first, then subtract any blacklist entries.
### Optional: `requires_tool` alias
[`docs/SKILLS.md`](../docs/SKILLS.md:439) documents a repeatable
`["requires_tool", "<name>"]` tag. Support it as an alias for the same policy:
if a skill has one or more `requires_tool` tags, union them into the allowlist.
This is additive and low-risk. (Note: `find_tag_value_string()` returns only the
first match, so a new helper is needed to collect all `requires_tool` values.)
### Where filtering happens
Add a pure function in [`src/tools/tools_schema.c`](../src/tools/tools_schema.c:1):
```c
/* Return a new schema array containing only tools named in `policy_csv`.
* policy_csv == NULL, "" or "*"/"all" returns the full schema (all tools).
* Unknown names are skipped. Caller frees the returned string. */
char* tools_build_openai_schema_json_filtered(const tools_context_t* ctx,
const char* policy_csv);
```
Implementation: build the full schema, parse it, iterate the array, and drop any
entry whose `function.name` is not in the allowlist. Short-circuit to the full
schema when the policy is empty or a wildcard. This mirrors the existing
[`build_sandboxed_tools_json_local()`](../src/tools/tool_skill.c:718) pattern, so
the approach is already proven in the codebase.
A shared helper `tool_allowed_by_policy(name, policy_csv)` should back both the
schema filter and the execution guard, so the two can never disagree. It returns
1 when the policy is empty/wildcard, or when `name` is in the list.
### Threading the policy to execution
1. Change the signature of [`agent_on_trigger()`](../src/agent.c:1771) to accept
the policy:
```c
void agent_on_trigger(const char* skill_d_tag,
const char* skill_content,
const char* tools_policy, /* NEW */
cJSON* triggering_event,
const char* relay_url);
```
2. In [`execute_llm_action()`](../src/trigger_manager.c:921), pass
`t->tools_policy`:
```c
agent_on_trigger(t->skill_d_tag, t->skill_content, t->tools_policy, event, relay_url);
```
3. In `agent_on_trigger()`, replace the schema build:
```c
char* tools_json = tools_build_openai_schema_json_filtered(&g_tools_ctx, tools_policy);
```
4. Update the forward declaration in [`trigger_manager.c`](../src/trigger_manager.c:16)
and any other callers.
### Defense in depth: enforce at execution
Filtering the schema is the primary control, but the LLM could still emit a tool
call for a name not in the schema. Add a guard in the trigger tool loop
([`agent.c`](../src/agent.c:1910)) before `tools_execute()`:
```c
if (!tool_allowed_by_policy(tc->name, tools_policy)) {
tool_result = strdup("{\"success\":false,\"error\":\"tool not permitted by skill policy\"}");
} else {
tool_result = tools_execute(&g_tools_ctx, tc->name, tc->arguments_json);
}
```
This makes the policy a real boundary, not just a prompt hint — the same
principle [`plans/swarm.md`](swarm.md:258) calls for: *"runtime-enforced
per-execution permissions — not just prompt instructions or schema filtering."*
## Scope decisions
- **Trigger path only (first cut).** The DM path
([`agent.c`](../src/agent.c:2141)) and the HTTP API path
([`http_api.c`](../src/http_api.c:1007)) keep the full schema. DM already has
sender-tier gating; the API is admin-only. Filtering those is a follow-up.
- **Compose with `skill_run` sandbox.** [`execute_skill_run()`](../src/tools/tool_skill.c:2357)
already filters for external skills. Leave it; the new policy applies to the
trigger path. A later pass can intersect the two.
- **`tools_policy` is 256 bytes.** A long allowlist could truncate. Either raise
the buffer or validate length at parse time and warn. Note this in the change.
## Files to change
| File | Change |
|------|--------|
| [`src/tools/tools_schema.c`](../src/tools/tools_schema.c:2574) | Add `tools_build_openai_schema_json_filtered()` |
| [`src/tools/tools.h`](../src/tools/tools.h:30) | Declare the new function |
| [`src/agent.c`](../src/agent.c:1771) | `agent_on_trigger()` takes `tools_policy`; use filtered schema; add execution guard |
| [`src/trigger_manager.c`](../src/trigger_manager.c:16) | Update forward decl; pass `t->tools_policy` |
| [`src/trigger_manager.h`](../src/trigger_manager.h:52) | (optional) raise `tools_policy` size |
| [`docs/TOOLS.md`](../docs/TOOLS.md:745) | Correct the doc to describe the `tools` tag (not just `requires_tool`) |
## Verification
1. **Unit-ish:** `--dump-schemas` still lists all 92 (no policy). Add a debug
path or log line showing the filtered count for a trigger.
2. **Swarm test:** restart the workers, post a "my queen" question, and confirm
in the worker debug log that the LLM request's tool list contains only the
five swarm tools — no `nostr_post`.
3. **Behavioral:** confirm workers now call `swarm_claim` (not `nostr_post`) and
that claims carry `swarm-task` tags.
4. **Regression — no tag:** a skill with no `tools` tag still gets all tools.
5. **Wildcard:** a skill with `["tools", "*"]` gets all tools (same as no tag).
6. **Allowlist:** a skill with `["tools", "swarm_read,swarm_claim"]` gets exactly
those two.
7. **Unknown name:** `["tools", "swarm_read,bogus"]` gets `swarm_read` and logs a
warning for `bogus`.
8. **Execution guard:** a forced tool call for a non-allowlisted tool returns
`tool not permitted by skill policy` rather than executing.
## Risks
- **Over-restriction.** A skill that lists too few tools will fail tasks it
previously handled. Mitigation: the "no tag = all tools" fallback, plus a
clear error when a blocked tool is called.
- **Truncation.** The 256-byte `tools_policy` buffer. Mitigation: raise it or
validate.
- **Silent typos.** An unknown tool name is skipped, so a misspelled tool
silently disappears. Mitigation: log a warning listing unknown names.
## Follow-ups (out of scope)
- Apply policy to the DM and HTTP API paths.
- Intersect skill policy with the `skill_run` sandbox allowlist.
- Support a deny-list form (e.g. `-local_shell_exec`) if needed.
- Surface the effective tool set in `/api/status` or `trigger_list` for
observability.