client/plans/strudel-ai-integration.md

Strudel AI Integration Plan

Overview

Add an AI assistant to strudel.html that lets the user describe music in natural language and have the AI write/modify Strudel code in the editor.

Confirmed Design Decisions

• API format: OpenAI-compatible endpoint (covers OpenAI, OpenRouter, Ollama, etc.)
• AI behavior: Auto-detect whether to modify existing code or write new code
• Chat panel: Always visible, below the editor
• Conversation history: Kept in memory per session; "Clear Chat" button in sidenav resets it
• API key storage: localStorage (visible, acceptable for personal use)
• Auto-play after AI edit: No — user presses Play/Reload manually
• System prompt size: ~500 tokens (modern models already know Strudel)

Layout

strudel-container
├── strudel-controls (Play/Reload buttons) ← top
├── strudel-editor-section (Strudel editor)
└── strudel-ai-panel (AI chat) ← always visible, below editor
    ├── strudelAIHistory (scrollable, messages grow upward)
    └── strudelAIInput (textarea + Send button, pinned to bottom)

divSideNav
├── divSideNavHeader
├── divSideNavBody
│   └── divAISettings ← NEW
│       ├── API URL input
│       ├── API Key input (password)
│       ├── Model name input
│       ├── Save button
│       └── Clear Chat button
├── divRelaySection
└── divVersionBar

Files to Create/Modify

New: www/js/strudel-ai.mjs

Exports:

• getAISettings() — reads from localStorage
• saveAISettings(url, key, model) — writes to localStorage
• buildMessages(history, currentCode, userInput) — constructs messages array for API
• callAI(messages, settings) — fetch to OpenAI-compatible endpoint
• extractCode(responseText) — detects and extracts Strudel code from response
• isCodeResponse(text) — returns true if response looks like Strudel code

Modified: www/strudel.html

1. Sidenav — add divAISettings section in divSideNavBody
2. Body — add strudelAIPanel div below strudel-editor-section
3. JavaScript — import strudel-ai.mjs, wire up chat input, handle responses

Modified: www/css/strudel.css

Add styles for:

• .strudel-ai-panel — container, flex column
• .strudel-ai-history — scrollable, flex-grow
• .strudel-ai-input — row with textarea and button
• .ai-message — base message style
• .ai-message-user — right-aligned or distinct style
• .ai-message-assistant — left-aligned or distinct style
• .ai-message-error — red error messages
• #divAISettings — sidenav section styling

System Prompt

You are a Strudel live coding assistant. Strudel is a JavaScript port of TidalCycles for making music in the browser.

Rules:
- If the user asks you to create or modify music/sounds, respond with ONLY valid Strudel JavaScript code. No markdown fences, no explanation.
- If the user asks a question or wants an explanation, respond in plain text only. Do not include code.
- When modifying, preserve the overall structure unless asked to change it completely.

Current code in the editor:
{{CURRENT_CODE}}

Code Detection Logic

A response is treated as Strudel code if it contains any of:

• note(, s(, sound(, stack(, samples(, n(, chord(
• .sound(, .s(, .note(
• setcps(, setCps(

Otherwise treated as a text response (shown in chat only, editor unchanged).

API Call

fetch(`${apiUrl}/chat/completions`, {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': `Bearer ${apiKey}`
  },
  body: JSON.stringify({
    model: model,
    messages: messages,  // [{role, content}, ...]
    temperature: 0.7,
    max_tokens: 1000
  })
})

Conversation History Format

// In-memory array, reset by "Clear Chat"
let conversationHistory = [];

// Each exchange adds two entries:
conversationHistory.push({ role: 'user', content: userInput });
conversationHistory.push({ role: 'assistant', content: aiResponse });

// buildMessages() prepends the system message:
[
  { role: 'system', content: systemPrompt },
  ...conversationHistory,
  { role: 'user', content: currentUserInput }
]

localStorage Keys

• strudel-ai-url — API base URL (e.g. https://api.openai.com/v1)
• strudel-ai-key — API key
• strudel-ai-model — model name (e.g. gpt-4o, claude-3-5-sonnet)

Error Handling

• Network error → show in chat as error message, keep current editor code
• API error (4xx/5xx) → show error message with status code in chat
• Invalid/empty response → show "AI returned an empty response" in chat
• Code injection error → show error in chat, keep previous code

Implementation Steps

1. Create www/js/strudel-ai.mjs with all helper functions
2. Add AI settings HTML to sidenav divSideNavBody
3. Add AI chat panel HTML below editor in divBody
4. Add CSS for chat panel and sidenav settings
5. Import strudel-ai.mjs in strudel.html script
6. Initialize AI settings from localStorage on page load
7. Wire Save button in sidenav
8. Wire Clear Chat button (resets conversationHistory array, clears chat UI)
9. Wire Send button and Enter key on textarea
10. Implement send handler: build messages → call AI → handle response
11. Commit and push