Claude Code Integration

When Envoy starts, it generates a complete Claude Code configuration in your project root. This gives Claude Code deep context about TouchDesigner development patterns, your project structure, and the MCP tools available through Envoy.

Generated Files

File/Directory	Purpose	Regenerated on start?
AGENTS.md	Universal AI instructions (always written, whichever clients are selected)	Yes
CLAUDE.md	Project context and critical rules	Yes -- or merged into, if the file is yours (see below)
.mcp.json	MCP server connection config	Yes
.embody/envoy-bridge.py	STDIO-to-HTTP bridge for MCP transport	Yes
.claude/settings.local.json	Tool permissions and MCP server config	No — content follows your Tool Permissions choice; merges into an existing file, preserving your other keys
.claude/rules/	Always-loaded conventions (see below)	Yes (unless edited)
.claude/skills/	On-demand workflow guides (see below)	Yes (unless edited)

AGENTS.md, CLAUDE.md, .claude/rules/, and .claude/skills/ are committed (not gitignored); the runtime files (.mcp.json, .embody/, .claude/settings.local.json) are automatically added to .gitignore.

If you already have a CLAUDE.md

Embody merges into it rather than replacing or skipping it. Your content is left exactly as written, and Embody maintains one delimited block inside the file -- opened by a visible ## Embody / Envoy -- auto-generated section heading that says which half is which and where your own edits belong. Later deploys refresh that block in place, wherever you moved it. Uninstall removes only the block and leaves the rest of your file.

Earlier versions instead wrote the guidance to a separate ENVOY.md. That preserved your file but delivered the instructions nowhere, because Claude Code does not auto-load ENVOY.md. Embody no longer writes that file; an ENVOY.md left over from an older version is not touched, and you can delete it once the block is present in your CLAUDE.md.

The Yes (unless edited) files are refreshed from their templates on start only while pristine. Embody stamps each file it generates with a hash of its own content, in the <!-- Generated by Embody/Envoy ... sha:... --> marker line; once you edit a generated rule or skill the hash no longer matches, your version is preserved, and Embody says so in the log — delete the file to opt back into regeneration. Because the hash rides inside the file, that protection survives a clone: a teammate who pulls your edited rule keeps it too. See How It Works.

Rules (Always-Loaded)

Rules are loaded into every Claude Code conversation automatically. They provide conventions that prevent common mistakes when working with TouchDesigner.

Rule	What it covers
network-layout.md	Always-loaded layout invariants (200-unit grid, size+gap spacing, forward wire flow, docked-op hugging, annotation coverage); the full positioning recipe lives in /create-operator
td-python.md	Parameter access (.eval() vs .val), operator path portability, threading, cook model
mcp-safety.md	Thread boundary (never access TD from background thread), localhost binding, 30s timeout
parameters.md	Custom parameter design: value access, required help text, section breaks, ordering, pages, naming, and styles
performance.md	Performance gating protocol, stop conditions, and a safe-caps summary; the full crash-cause and safe-default-caps tables live in /td-api-reference (Heavy-Build Safety)
td-connectivity.md	Session-start connectivity checks, the bridge reconciler and Envoy liveness watchdog, and manual recovery steps
multi-session.md	The session-start presence check, _peers advisories and gate hard stops, and claiming a scope before big or destructive work
worktree-td-safety.md	When to isolate multi-step edits in a git worktree instead of the live tree, and the discipline for landing that diff back
tdxn-economy.md	Fewer operators, parameters and lines in every network that ends up in a .tdxn: measure the file by bucket, read parameters with a Parameter CHOP, share identical GLSL, let defaults carry the common case, verify a cut with a frame diff

Skills (On-Demand)

Skills are loaded only when needed, keeping the context window lean. Claude Code loads them automatically before performing the relevant operation.

Skill	Trigger
/create-operator	Before creating or moving any operator (create_op, copy_op, set_op_position, or execute_python builds); carries the canonical positioning recipe
/create-extension	Before creating TD extensions via create_extension
/debug-operator	When diagnosing operator errors
/externalize-operator	Before tagging or saving externalizations
/manage-annotations	Before creating or modifying annotations
/td-api-reference	Before writing TD Python code, and before any heavy build (its Heavy-Build Safety section holds the crash-cause and safe-caps tables)
/mcp-tools-reference	Before the first MCP call in a session
/visual-aesthetics	Before building or refining any rendered visual output (generative art, VJ visuals, shaders, scenes, renders)
/pop-networks	Before POP networks, particle systems, or GPU point/geometry work
/movie-export	Before recording or exporting any movie or image sequence
/parameter-design	Before creating or designing custom parameters on a COMP
/td-recovery	When Envoy/TD connectivity has not self-healed after ~15s
/multi-session-etiquette	The moment a _peers advisory or a second AI session appears
/merge-divergent-tox	When a merge or rebase conflicts on a .tox/.toe, or when comparing a component across branches, machines, or TD builds
/brief	User-invoked: /brief <request> compiles a conversational ask into a task brief in briefs/ (skills to load, anchors, success criteria, gates) that the work then executes from
/glsl-shaders	Before writing GLSL or creating a glslTOP, glslmultiTOP, glslMAT or glslPOP: docked DATs, uniforms and the Colors page, TD's built-in functions, feedback and multi-pass, compile-error reading
/operator-gotchas	Before wiring TOP, CHOP, MAT, DAT, light, camera or instancing operators you have not built before: abbreviated parameter names, defaults that silently misbehave, input-order rules, all verified live
/testing	Before declaring a build, fix or show done, and before any soak or performance test: the verification ladder, run_soak_test and the observer-effect rules, iterate-by-capture, end-to-end rehearsal
/collab	User-invoked: thinking-partner mode -- one small step at a time in the visible output, the next decision handed back; /brief returns to the autonomous default

Each skill contains step-by-step workflows, API details, and common pitfalls specific to that operation.

Five skills (td-api-reference, parameter-design, mcp-tools-reference, visual-aesthetics, merge-divergent-tox) keep their long tables in a references/ folder beside SKILL.md, seven files in all, so the skill itself stays short and loads cheaply. Envoy writes the reference files with the skill, and get_guidance serves one as <skill>/<file> (for example visual-aesthetics/look-recipes).

Task Briefs (/brief)

For substantive, multi-step asks, Claude Code can compile your conversational request into a task brief before building — a short markdown contract that states, up front, everything a good build needs:

• Skills to load — which of the skills above must be loaded before which tool call
• Anchors — where the work happens, discovered from your live project (real container paths, never guesses)
• Success criteria — only verifiable statements: a captured and assessed output frame, clean get_op_errors, fps within tolerance of the pre-build baseline
• Gates — a performance baseline before building, multi-session claims, and flagged destructive steps that wait for your confirmation

Two ways to invoke it:

/brief make the gallery wall react to the music

Everything after /brief is the request — as vague and conversational as you like; turning it into a precise contract is the skill's job. Or type a bare /brief after describing what you want in chat, and it formalizes the conversation's current ask.

The compiled brief lands in briefs/<date>-<slug>.md at your project root (gitignored — Embody manages the entry). Claude presents a summary for your approval, then executes from the brief: skills loaded at the stated moments, gates honored, every success criterion checked before "done." Because the brief is a file rather than chat history, it outlives the session — a fresh session or a sub-agent can pick it up and execute it without ever having seen the conversation.

Trivial asks skip the ritual: a one-parameter change or a quick question doesn't need a brief, and the skill says so rather than taxing small requests. When a build goes wrong despite a brief, the miss is recorded in the brief's Deviations section and fed back into the skill — briefs get better the more you use them.

/brief is user-invoked only: Claude won't run it on its own, though it may suggest it when a big ask arrives without one.

STDIO Bridge

Claude Code connects to Envoy through a STDIO bridge script (.embody/envoy-bridge.py) that translates between Claude Code's STDIO transport and Envoy's HTTP endpoint. The bridge provides four meta-tools that work even when TouchDesigner is not running:

Tool	Description
get_td_status	Check if TD is running, whether Envoy is reachable, crash detection, restart attempts remaining, and instance registry status
launch_td	Launch TD with the project's .toe file and wait for Envoy to become reachable. On fresh clones (where .embody/envoy.json's td_executable path doesn't exist locally), the bridge auto-picks a TouchDesigner install — the machine-local .embody/local.json build pin when present, the newest installed TD otherwise — see Architecture.
restart_td	Gracefully quit TD, then relaunch and wait for Envoy. Targets only this session's pinned instance's verified process (unpinned: the registry default's) — on machines running several TD projects, the others are never touched
switch_instance	List all registered TD instances or switch the bridge to a different running instance
list_dialogs	List the modal dialogs blocking TD (message boxes, missing-file prompts, the license box), with optional screenshots, even while TD's main thread is frozen
dismiss_dialog	Dismiss a blocking dialog (close, then escape, then enter, verified gone) so the session can continue without a human at the machine

This means Claude Code can start a TD session from scratch — no need to manually open TouchDesigner first. If TD crashes, Claude can detect it and restart automatically.

The bridge also handles crash-loop protection (max 3 launches in 5 minutes), automatic retry with backoff on transient connection failures, and orphan process cleanup when Claude Code exits.

Working with Multiple Instances

If you have multiple TouchDesigner instances running with Envoy enabled (e.g., your main project and a test project), the bridge connects to one at a time. Use switch_instance to move between them:

• List instances: Call switch_instance with no arguments to see all registered instances and their reachability
• Switch: Call switch_instance with the instance name (.toe filename without the extension) to redirect all subsequent MCP calls to that instance
• Address one call: Pass instance=<name> on any Envoy tool (every tool advertises the optional argument) to route just that call to another registered instance without switching — inspect the show file while pinned to the dev one, then carry on

Each instance gets its own port automatically (ports 9870–9879). Switching is instant — no restart required.

See Architecture for technical details.

How It Works

Embody stores master copies of all rules and skills as template DATs inside the templates baseCOMP. When Envoy starts in a user project, _extractClaudeConfig() reads these templates and writes them to the project's .claude/ directory. This means:

• Updates are automatic — upgrading Embody gives you the latest rules and skills
• Templates are the source of truth for pristine files — an unedited generated .claude/ file is refreshed from its template on Envoy start, but Embody stamps each file it generates with a hash of its own content, in the <!-- Generated by Embody/Envoy ... sha:... --> marker line and will not overwrite one you've since edited (delete the file to opt back into regeneration). The stamp travels with the file, so the protection holds on a clone, a fresh checkout, or any machine that has never run Embody
• Project-specific customization — add your own rules or skills to .claude/ alongside the generated ones (they won't be overwritten)

Live Build Visualization

Two independent toggles on the Embody COMP's Envoy page control this, both ON by default:

• Envoy Follow (Envoyfollow) — the camera follows the operator Envoy is working on, panning the network editor to it.
• Embot (Embotenable) — the builder-bot mascot appears on each operator Envoy touches and narrates what it does.

While the agent works through Envoy, with Envoy Follow on:

• Within the network you're viewing, the editor smoothly glides to center on each operator just touched (ease-out, one step per frame).
• When the work moves to a COMP no pane is showing, the editor navigates a network-editor pane into that COMP and snaps to frame the op — you can't glide across networks (different coordinate spaces), so it cuts.

With Embot on:

• A small builder-bot ("embot") — a figure made of minimal network-box annotations — hops between the nodes being worked on, hovers when idle, and throws an occasional gesture (a wave, a reach, a pump, the odd robot dance). Its color tracks "thinking time": cool cyan-green right after Envoy acts, warming toward red the longer the gap. The node Envoy just touched pulses the Envoy accent.

Embot settles rather than chases: rebuilding him in a new network is real main-thread work, so when the agent is hopping between networks he stays where the work actually lives, keeps hopping between the operators there, and keeps narrating the current op from where he stands. He relocates once the work has genuinely settled somewhere new (or a queued batch proves it is about to), and if the work never settles anywhere he still moves in after a few seconds rather than standing on a stale node forever. The node-colour pulse still fires on every operator Envoy touches, in every network. With Embot off, Envoy Follow cuts to every hop exactly as before.

With Envoy Follow off but Embot on, he appears only in the network you are already viewing — building nine annotations into a network the camera will never travel to costs real frame time for something nobody can see. Navigate to where the agent is working and he arrives there.

It yields the instant you pan, zoom, or navigate the view yourself, and resumes only once you stop — it never yanks the view mid-interaction. The bot and pulse retire after a stretch of quiet.

The agent never sees him either: Envoy's read tools (get_annotations, get_network_layout, query_network, find_children, get_enclosed_ops) filter his parts out and report how many they hid as embot_hidden, so nine annotations that belong to no .tdxn can't be mistaken for part of your network.

This is purely a viewing aid: it writes only pane/view state (which TouchDesigner never externalizes), the bot is destroyed before every save, and it runs entirely on the main thread, so it adds nothing to your saved files and never affects a build. Leave it off if you'd rather your view never move on its own.

Customization

You can extend the generated configuration:

• Add project-specific rules: Create additional .md files in .claude/rules/ — Claude Code loads all rules in this directory
• Add custom commands: Create .md files in .claude/commands/ with prompt instructions
• Modify permissions: Set the Tool Permissions parameter on Embody's Envoy page (or re-run the setup wizard) to choose how much Embody auto-approves — all tools, read-only tools only, nothing, or leave the file untouched. See MCP Tool Permissions. You can also hand-edit .claude/settings.local.json to allow or restrict specific tools.

Tip

You can edit the Envoy-generated rules and skills directly — Embody stamps each file it generates with a hash of its own content, in the <!-- Generated by Embody/Envoy ... sha:... --> marker line, so it won't overwrite one you've changed (it logs that it kept your edits). Commit your edited file and it stays protected everywhere: the hash is inside the file, not in a machine-local sidecar. Pristine generated files still refresh on regeneration; delete a file to discard your changes and pull the latest template.