Claude Code Integration

When Envoy starts, it generates a complete Claude Code configuration in your project root. This gives Claude Code deep context about TouchDesigner development patterns, your project structure, and the MCP tools available through Envoy.

Generated Files

File/Directory	Purpose	Regenerated on start?
AGENTS.md	Universal AI instructions (always written, any AI Client)	Yes
CLAUDE.md	Project context and critical rules	Yes
.mcp.json	MCP server connection config	Yes
.embody/envoy-bridge.py	STDIO-to-HTTP bridge for MCP transport	Yes
.claude/settings.local.json	Tool permissions and MCP server config	No — content follows your Tool Permissions choice; merges into an existing file, preserving your other keys
.claude/rules/	Always-loaded conventions (see below)	Yes (unless edited)
.claude/skills/	On-demand workflow guides (see below)	Yes (unless edited)

AGENTS.md, CLAUDE.md, .claude/rules/, and .claude/skills/ are committed (not gitignored); the runtime files (.mcp.json, .embody/, .claude/settings.local.json) are automatically added to .gitignore.

The Yes (unless edited) files are refreshed from their templates on start only while pristine. Embody records a content hash of every file it generates (in .embody/generated-hashes.json); once you edit a generated rule or skill, your version is preserved and not overwritten — delete the file to opt back into regeneration. See How It Works.

Rules (Always-Loaded)

Rules are loaded into every Claude Code conversation automatically. They provide conventions that prevent common mistakes when working with TouchDesigner.

Rule	What it covers
network-layout.md	Always-loaded layout invariants (200-unit grid, size+gap spacing, forward wire flow, docked-op hugging, annotation coverage); the full positioning recipe lives in /create-operator
td-python.md	Parameter access (.eval() vs .val), operator path portability, threading, cook model
mcp-safety.md	Thread boundary (never access TD from background thread), localhost binding, 30s timeout
parameters.md	Custom parameter design: value access, required help text, section breaks, ordering, pages, naming, and styles
performance.md	Performance gating protocol, stop conditions, and a safe-caps summary; the full crash-cause and safe-default-caps tables live in /td-api-reference (Heavy-Build Safety)
td-connectivity.md	Session-start connectivity checks, the bridge reconciler and Envoy liveness watchdog, and manual recovery steps
multi-session.md	The session-start presence check, _peers advisories and gate hard stops, and claiming a scope before big or destructive work
worktree-td-safety.md	When to isolate multi-step edits in a git worktree instead of the live tree, and the discipline for landing that diff back

Skills (On-Demand)

Skills are loaded only when needed, keeping the context window lean. Claude Code loads them automatically before performing the relevant operation.

Skill	Trigger
/create-operator	Before creating or moving any operator (create_op, copy_op, set_op_position, or execute_python builds); carries the canonical positioning recipe
/create-extension	Before creating TD extensions via create_extension
/debug-operator	When diagnosing operator errors
/externalize-operator	Before tagging or saving externalizations
/manage-annotations	Before creating or modifying annotations
/td-api-reference	Before writing TD Python code, and before any heavy build (its Heavy-Build Safety section holds the crash-cause and safe-caps tables)
/mcp-tools-reference	Before the first MCP call in a session
/visual-aesthetics	Before building or refining any rendered visual output (generative art, VJ visuals, shaders, scenes, renders)
/pop-networks	Before POP networks, particle systems, or GPU point/geometry work
/movie-export	Before recording or exporting any movie or image sequence
/parameter-design	Before creating or designing custom parameters on a COMP
/td-recovery	When Envoy/TD connectivity has not self-healed after ~15s
/multi-session-etiquette	The moment a _peers advisory or a second AI session appears
/merge-divergent-tox	When a merge or rebase conflicts on a .tox/.toe, or when comparing a component across branches, machines, or TD builds
/brief	User-invoked: /brief <request> compiles a conversational ask into a task brief in briefs/ (skills to load, anchors, success criteria, gates) that the work then executes from

Each skill contains step-by-step workflows, API details, and common pitfalls specific to that operation.

Task Briefs (/brief)

For substantive, multi-step asks, Claude Code can compile your conversational request into a task brief before building — a short markdown contract that states, up front, everything a good build needs:

• Skills to load — which of the skills above must be loaded before which tool call
• Anchors — where the work happens, discovered from your live project (real container paths, never guesses)
• Success criteria — only verifiable statements: a captured and assessed output frame, clean get_op_errors, fps within tolerance of the pre-build baseline
• Gates — a performance baseline before building, multi-session claims, and flagged destructive steps that wait for your confirmation

Two ways to invoke it:

/brief make the gallery wall react to the music

Everything after /brief is the request — as vague and conversational as you like; turning it into a precise contract is the skill's job. Or type a bare /brief after describing what you want in chat, and it formalizes the conversation's current ask.

The compiled brief lands in briefs/<date>-<slug>.md at your project root (gitignored — Embody manages the entry). Claude presents a summary for your approval, then executes from the brief: skills loaded at the stated moments, gates honored, every success criterion checked before "done." Because the brief is a file rather than chat history, it outlives the session — a fresh session or a sub-agent can pick it up and execute it without ever having seen the conversation.

Trivial asks skip the ritual: a one-parameter change or a quick question doesn't need a brief, and the skill says so rather than taxing small requests. When a build goes wrong despite a brief, the miss is recorded in the brief's Deviations section and fed back into the skill — briefs get better the more you use them.

/brief is user-invoked only: Claude won't run it on its own, though it may suggest it when a big ask arrives without one.

STDIO Bridge

Claude Code connects to Envoy through a STDIO bridge script (.embody/envoy-bridge.py) that translates between Claude Code's STDIO transport and Envoy's HTTP endpoint. The bridge provides four meta-tools that work even when TouchDesigner is not running:

Tool	Description
get_td_status	Check if TD is running, whether Envoy is reachable, crash detection, restart attempts remaining, and instance registry status
launch_td	Launch TD with the project's .toe file and wait for Envoy to become reachable. On fresh clones (where .embody/envoy.json's td_executable path doesn't exist locally), the bridge auto-picks a TouchDesigner install — the machine-local .embody/local.json build pin when present, the newest installed TD otherwise — see Architecture.
restart_td	Gracefully quit TD, then relaunch and wait for Envoy. Targets only the active instance's verified process — on machines running several TD projects, the others are never touched
switch_instance	List all registered TD instances or switch the bridge to a different running instance

This means Claude Code can start a TD session from scratch — no need to manually open TouchDesigner first. If TD crashes, Claude can detect it and restart automatically.

The bridge also handles crash-loop protection (max 3 launches in 5 minutes), automatic retry with backoff on transient connection failures, and orphan process cleanup when Claude Code exits.

Working with Multiple Instances

If you have multiple TouchDesigner instances running with Envoy enabled (e.g., your main project and a test project), the bridge connects to one at a time. Use switch_instance to move between them:

• List instances: Call switch_instance with no arguments to see all registered instances and their reachability
• Switch: Call switch_instance with the instance name (.toe filename without the extension) to redirect all subsequent MCP calls to that instance

Each instance gets its own port automatically (ports 9870–9879). Switching is instant — no restart required.

See Architecture for technical details.

How It Works

Embody stores master copies of all rules and skills as template DATs inside the templates baseCOMP. When Envoy starts in a user project, _extractClaudeConfig() reads these templates and writes them to the project's .claude/ directory. This means:

• Updates are automatic — upgrading Embody gives you the latest rules and skills
• Templates are the source of truth for pristine files — an unedited generated .claude/ file is refreshed from its template on Envoy start, but Embody records a content hash of each file it generates (.embody/generated-hashes.json) and will not overwrite one you've since edited (delete the file to opt back into regeneration)
• Project-specific customization — add your own rules or skills to .claude/ alongside the generated ones (they won't be overwritten)

Live Build Visualization

Two independent toggles on the Embody COMP's Envoy page control this, both ON by default:

• Envoy Follow (Envoyfollow) — the camera follows the operator Envoy is working on, panning the network editor to it.
• Embot (Embotenable) — the builder-bot mascot appears on each operator Envoy touches and narrates what it does.

While the agent works through Envoy, with Envoy Follow on:

• Within the network you're viewing, the editor smoothly glides to center on each operator just touched (ease-out, one step per frame).
• When the work moves to a COMP no pane is showing, the editor navigates a network-editor pane into that COMP and snaps to frame the op — you can't glide across networks (different coordinate spaces), so it cuts.

With Embot on:

• A small builder-bot ("embot") — a figure made of minimal network-box annotations — hops between the nodes being worked on, hovers when idle, and throws an occasional gesture (a wave, a reach, a pump, the odd robot dance). Its color tracks "thinking time": cool cyan-green right after Envoy acts, warming toward red the longer the gap. The node Envoy just touched pulses the Envoy accent.

Embot settles rather than chases: rebuilding him in a new network is real main-thread work, so when the agent is hopping between networks he stays where the work actually lives, keeps hopping between the operators there, and keeps narrating the current op from where he stands. He relocates once the work has genuinely settled somewhere new (or a queued batch proves it is about to), and if the work never settles anywhere he still moves in after a few seconds rather than standing on a stale node forever. The node-colour pulse still fires on every operator Envoy touches, in every network. With Embot off, Envoy Follow cuts to every hop exactly as before.

With Envoy Follow off but Embot on, he appears only in the network you are already viewing — building nine annotations into a network the camera will never travel to costs real frame time for something nobody can see. Navigate to where the agent is working and he arrives there.

It yields the instant you pan, zoom, or navigate the view yourself, and resumes only once you stop — it never yanks the view mid-interaction. The bot and pulse retire after a stretch of quiet.

This is purely a viewing aid: it writes only pane/view state (which TouchDesigner never externalizes), the bot is destroyed before every save, and it runs entirely on the main thread, so it adds nothing to your saved files and never affects a build. Leave it off if you'd rather your view never move on its own.

Customization

You can extend the generated configuration:

• Add project-specific rules: Create additional .md files in .claude/rules/ — Claude Code loads all rules in this directory
• Add custom commands: Create .md files in .claude/commands/ with prompt instructions
• Modify permissions: Set the Tool Permissions parameter on Embody's Envoy page (or re-run the setup wizard) to choose how much Embody auto-approves — all tools, read-only tools only, nothing, or leave the file untouched. See MCP Tool Permissions. You can also hand-edit .claude/settings.local.json to allow or restrict specific tools.

Tip

You can edit the Envoy-generated rules and skills directly — Embody records a content hash of each generated file (.embody/generated-hashes.json) and won't overwrite one you've changed (it logs that it kept your edits). Pristine generated files still refresh on regeneration; delete a file to discard your changes and pull the latest template.