0004 — Code mode: MCP tools driven through Monty

Status: accepted

Date: 2026-11-03

Context

OpenAiCompatibleBackend exposes every MCP tool to the model as an OpenAI function. The model calls one tool per round, the full result enters the conversation, and the loop allows 8 rounds.

That shape fails on Kubernetes data:

• kubectl get ... -o yaml for a busy namespace is tens to hundreds of kilobytes. The answer the user wants is usually a handful of names or a count.
• The MCP client capped every result at 64k characters. get_api_resources produced ~21 KB of YAML, a 12k cap cut it mid-document, and the model and the verbose logs both received invalid YAML. Raising the cap only moves the failure.
• A dependent sequence (list workspaces, then list objects in each, then get one) costs one model round trip per step, and every intermediate result is paid for in context.

Pydantic AI solves this with CodeMode, backed by Monty, a sandboxed Python subset written in Rust:

Runs Monty behind CodeModeToolset. Instead of making sequential tool calls, the model writes Python that calls your tools as functions, and Monty executes it: one round trip for a task that would otherwise take three.

The model writes a program that fans out tool calls with asyncio.gather, filters and aggregates the results in Python, and returns only what matters. Large intermediate results stay inside the sandbox.

Monty ships as Rust crates (monty, monty-types, 1.4.0 on crates.io, MIT, MSRV 1.96). MontyRepl keeps globals between snippets and pauses execution at every call to a host function, so the host can run the real call and resume.

Decision

1. Every MCP tool call goes through one model-facing tool, run_code. The model never sees the MCP tools as OpenAI functions; it sees them as Python async def stubs in the run_code description.
2. run_code executes the model's Python in an in-process Monty REPL owned by a dedicated OS thread. MCP calls made from that Python are dispatched by the async host to the rmcp clients.
3. MCP tool output is unbounded. The client no longer truncates. The only bound is on what run_code returns to the model (see Limits).
4. kcp tools emit JSON (kubectl -o json) so the sandbox can parse and filter everything with the stdlib. http_get returns JSON {status, content_type, body} with the full body. MCP results that parse as JSON arrive in Python as dict / list values; anything else arrives as str.
5. McpToolExecutor holds its clients without a lock, so calls gathered in Python run concurrently against the MCP servers.

run_code

Input schema:

{
  "type": "object",
  "properties": {
    "code": {"type": "string", "description": "Python to run"},
    "restart": {"type": "boolean", "description": "reset REPL state first"}
  },
  "required": ["code"]
}

The description carries three parts: the sandbox rules, the return contract, and the catalog of available functions.

Example program for "which socialweb.computer kinds exist in root:app?":

resources = await get_api_resources(kcp=KCP, workspace="root:app")
sorted({r["kind"] for r in resources if r["api_version"].startswith("socialweb.computer/")})

Parallel calls:

import asyncio
pods, events = await asyncio.gather(
    list_objects(kcp=KCP, workspace="root:app", resource="pods"),
    get_events(kcp=KCP, workspace="root:app"),
)
bad = [p["metadata"]["name"] for p in pods["items"]
       if p["status"].get("phase") != "Running"]
{"not_running": bad, "warnings": sum(1 for e in events["items"] if e["type"] == "Warning")}

Flow

sequenceDiagram
    participant M as model
    participant B as OpenAiCompatibleBackend
    participant C as CodeModeRun (async)
    participant T as Monty thread
    participant S as MCP servers

    M->>B: tool_call run_code {code}
    B->>C: call_tool("run_code", {code})
    C->>T: Job {code, restart}
    T->>T: feed_start(code)
    T-->>C: Call {call_id: 1, list_objects, kwargs}
    T->>T: resume_pending (future)
    T-->>C: Call {call_id: 2, get_events, kwargs}
    T->>T: resume_pending (future)
    T-->>C: NeedResults
    par concurrent
        C->>S: tools/call list_objects
        C->>S: tools/call get_events
    end
    S-->>C: JSON text (unbounded)
    C->>T: (1, dict) / (2, dict)
    T->>T: resume futures, filter in Python
    T-->>C: Complete {value, prints}
    C-->>B: compact JSON of the filtered value
    B->>M: tool message

Threads and channels

flowchart LR
    subgraph tokio
        B[backend start_run] -->|run_scope| R[CodeModeRun]
        R -->|FuturesUnordered| X[inner ToolExecutor<br/>McpToolExecutor]
    end
    subgraph monty thread
        E[engine loop<br/>Option&lt;MontyRepl&gt;]
    end
    R -- Job --> E
    E -- HostEvent::Call / NeedResults / Done --> R
    R -- call results --> E

monty uses Rc internally, so the REPL and its suspended snapshots never cross threads. The engine thread owns them for its whole life and talks to the async side over channels. CPU-bound Python never blocks a tokio worker.

Run scope

ToolExecutor gains fn run_scope(&self) -> Option<Arc<dyn ToolExecutor>>, which defaults to None. OpenAiCompatibleBackend::start_run calls it once per run. CodeModeToolExecutor returns a fresh CodeModeRun with its own engine thread and an empty REPL.

REPL state (variables, imports, functions) therefore persists across run_code calls within one agent turn, so the model can store a large result in a variable, inspect its shape, and filter it in a later call without refetching it. restart: true clears it. When the turn ends, the scope is dropped, the job channel closes, and the thread exits. Concurrent agents never share a REPL.

Engine loop

Monty progress	Host answer
NameLookup(name) with a catalog name	MontyObject::function(name, docstring)
NameLookup(name) otherwise	NotFound → NameError
FunctionCall for a catalog function, keyword args only	send Call to async side, resume_pending
FunctionCall with positional args	TypeError: use keyword arguments
FunctionCall past the nested-call budget	RuntimeError: nested tool call budget exceeded
ResolveFutures	send NeedResults, block for ≥1 result, resume with every result received
OsCall	PermissionError: os access disabled in asyncai code mode
Complete {repl, value}	keep repl, send Done(value, prints)
ReplStartError	keep the preserved repl, send the traceback as an error

Nested MCP results map into the sandbox as follows:

MCP result	Python value
is_error: true	raises RuntimeError(<text>); catchable
text (or structured_content when there is no text) that parses as JSON	the parsed value
other text	str
rmcp transport error	raises RuntimeError("tool error: ...")

Python stubs

The catalog renders each MCP tool's JSON schema as an async def stub with keyword-only parameters:

async def list_objects(*, kcp: str, workspace: str, resource: str, namespace: str | None = None, selector: str | None = None) -> Any:
    """List objects of one resource type in a workspace."""

JSON schema	Python
string	str
integer	int
number	float
boolean	bool
array of T	list[T]
object / absent / boolean schema	dict[str, Any] / Any
enum	Literal[...]
type list with null, or optional property	T | None = None

Required parameters come first. A tool name that is not a Python identifier is sanitized ([^A-Za-z0-9_] → _, leading digit gets a _ prefix, collisions get a numeric suffix). The engine maps the Python name back to the MCP name.

Value conversion

JSON	Python	back to JSON
null	None	null
bool	bool	bool
integer in i64	int	number
other number	float	number (NaN/inf → string)
string	str	string
array	list	array
object	dict	object
—	tuple, set, frozenset	array
—	dict with non-str keys	object, keys via repr
—	anything else	repr() string

Return contract

The last expression in the snippet is its result. print() output is collected separately.

Snippet ends with	run_code returns
a non-None value, nothing printed	the value
an assignment or None, nothing printed	(no output; the last expression was None) (A11)
an assignment or None, something printed	{"output": "<prints>"}
a non-None value, something printed	{"output": "<prints>", "result": <value>}
an exception	error result with the Python traceback, plus a Hint: for known Monty gaps (A11)
an un-awaited coroutine or gather	error: add await (A11)

A str result is returned as-is. Everything else is compact JSON.

Limits

Limit	Value	On breach
execution time per snippet (max_feed_duration)	30 s; excludes time awaiting tools	error result; REPL kept
heap	not capped in-process (see below)	—
recursion depth	Monty default (1000)	RecursionError
suspensions per snippet (max_suspensions)	10 000	uncatchable RuntimeError
nested tool calls per snippet	100	RuntimeError at the call site
collected print() output	1 MiB	MemoryError
MCP tool output	unbounded	—
run_code result to the model	20 000 chars	error result with size, a 2 000-char preview, and "assign to a variable and return a filtered subset"

The 20 000-char guard protects the context window, not the tools. The value is still in the REPL if the model assigned it, so the next snippet filters without refetching.

Monty enforces max_memory only when monty-alloc is the process's global allocator. That allocator caps the whole process, so in an agent it would count the tokio runtime, the MCP clients, and every buffer next to the sandbox. asyncai therefore does not set max_memory. A hard heap ceiling comes with the monty-pool follow-up, where each worker process installs monty-alloc for itself.

Limits are fields of CodeModeLimits, so library users can tune them.

Logging

With --verbose, NDJSON on stderr:

Event	Fields
model.tool_call	tool: "run_code", arguments: {code, restart?}
model.tool_result	tool, is_error, content (JSON-decoded when possible, otherwise truncated text)
mcp.tool_call	via: "code_mode", tool, arguments
mcp.tool_result	via: "code_mode", tool, is_error, bytes, content (JSON-decoded when possible, otherwise truncated text)

Session transcripts

Session harnesses record what the model saw: the run_code tool_use and its tool_result. Nested MCP calls are not model turns. They are not written as transcript records and do not enter the parentUuid chain; they appear in the verbose log only. The transcript order rule from ADR 0002 is unchanged.

Invariants

1. With --mcp-stdio, the model's tool list is exactly [run_code].
2. Every MCP call made for the model goes through the Monty engine.
3. The MCP client returns full tool output. No component between the MCP server and the sandbox truncates it.
4. The sandbox has no filesystem, environment, network, or subprocess access. Its only way out is a catalog function.
5. A snippet cannot block a tokio worker: Python runs on the engine thread.
6. A runaway snippet ends with an error result within its time budget; it never hangs the agent.
7. REPL state is scoped to one backend run and never shared across runs or agents.
8. kcp object tools emit JSON; get_api_resources emits a JSON array.
9. Tests use a fake ToolExecutor or the bundled stdio servers with fake backends. No live model, cluster, or atirc server.

Test plan

Unit (src/code_mode/):

• stub rendering: required/optional ordering, enum, array, nullable type list, boolean schema, name sanitizing and collision suffixes;
• JSON → Monty → JSON round trip, including non-str keys and tuples;
• return contract table;
• 20 000-char guard.

Integration (tests/code_mode.rs, fake in-memory executor):

• asyncio.gather of two tools plus filtering returns one small value; both inner calls recorded;
• a 1 MB JSON tool result is filtered down inside the sandbox;
• is_error tool result raises a catchable RuntimeError;
• unknown function → NameError in the error result;
• open(...) and os.getenv(...) → PermissionError;
• while True: pass → time-limit error, no hang;
• REPL state persists across two run_code calls in one scope; restart clears it; a new scope starts empty;
• positional arguments → TypeError;
• nested-call budget enforced.

Backend and CLI:

• mocked LLM requests list only run_code; a run_code tool call runs and the tool message carries the filtered result;
• asyncai --mcp-stdio asyncai-mcp-simple with a mocked model runs await http_get(url=...) against a loopback HTTP server; the Claude Code transcript has a run_code tool_use and tool_result.

kcp:

• fake kubectl sees -o json on object calls; get_api_resources returns a JSON array.

Live verification

In tmux pane 0:1.1, run the readwrite agent with --verbose, subscribe to the channel with atirc sub, and publish prompts with atirc pub:

1. "which socialweb.computer kinds exist in root:app?" → one model.tool_call run_code, a nested mcp.tool_call get_api_resources, a short reply.
2. "list the names of all configmaps in every namespace" → a large list_objects result filtered in the sandbox; the reply lists names only.
3. apply → get → patch → delete of a test ConfigMap through run_code.

Repeat 1–2 with socialweb-computer-kcp-readonly. Confirm kubectl invocations use -o json, and that the session transcript resumes with deepseek-claude -p --resume <uuid>.

Consequences

• One model round trip covers dependent and parallel tool sequences.
• Large tool output costs sandbox memory, not context.
• Models must write valid Monty Python. Errors return as tracebacks, and the model can retry within the 8-round loop. Small models explore: they print shapes, hit errors, and retry. The final round withdraws run_code and asks for an answer from the results so far, so exploration never ends a turn without a reply.
• Tools without an output schema render as -> Any. The description tells the model to inspect a small sample before relying on such a shape. The bundled servers declare schemas (Amendment A1).
• Tool results are JSON-shaped rather than YAML. Humans read the verbose logs, which were already JSON.
• In the in-process sandbox, a stack-overflow or allocator abort inside Monty would take the agent process down. The pool sandbox (Amendment A5) removes that risk.

Alternatives considered

• monty-pool only. This was deferred in the first version and is now the preferred sandbox (Amendment A5). In-process stays available because the pool needs a separately installed worker binary, and tests must not.
• Keep direct tool calls next to run_code. This contradicts "all MCP tool calls driven through Monty" and invites the model to bypass filtering. Library users can still pass McpToolExecutor directly to with_tool_executor.
• Raise the output cap. It only moves the failure, and it still pays for every byte in context.
• Dynamic catalog in system instructions (Pydantic's dynamic_catalog). Our tool set is fixed per process, so the catalog lives in the run_code description.

Amendments (2026-11-03): hardening

Live runs with a 27B local model showed the remaining failure modes:

• untyped results caused shape guessing (kcps[0] on a dict);
• every call was tried on every MCP server;
• unknown keyword arguments passed silently;
• a hung tool blocked for up to an hour;
• there was no crash isolation and no heap cap;
• writes left no trace in the session file;
• the model spent rounds discovering the kcp id.

These amendments supersede the matching parts above.

A1. Typed results

ToolDefinition carries the MCP outputSchema and annotations (read_only, destructive). Each stub's return annotation comes from the output schema. Its docstring gains a Returns: line with a compact shape sketch: at most 4 levels and 12 properties per object, then ....

async def list_kcps(*, refresh: bool | None = None) -> dict[str, Any]:
    """
    Discover running socialweb-computer kcp servers...
    Returns: {"kcps": [{"id": str, "kubeconfig": str, "server": str, "running": bool, ...}], "errors": [str]}
    """

The kcp servers and http_get declare output schemas and return structuredContent. MCP requires structuredContent to be an object, so array results are wrapped: list_workspaces → {"workspaces": [...]}, get_api_resources → {"resources": [...]}. get_logs and delete_object stay plain text, with no schema. Each tool description ends with a one-line Python example.

A2. Routing

McpToolExecutor lists every server's tools at connect time and builds a name → server table. A call goes to exactly one server. If two servers declare the same name, the first wins; the duplicate is recorded and logged as a warn at startup.

A3. Call validation

A shared dispatcher checks every nested call before it reaches the host:

Check	Python error
positional arguments	TypeError: f() takes keyword arguments only
unknown keyword	TypeError: f() got an unexpected keyword argument 'x'; accepted: a, b
missing required keyword	TypeError: f() missing required keyword argument 'x'
nested-call budget exhausted	RuntimeError

Optional keywords passed as None are dropped, matching the stub default.

A4. Timeouts

Limit	Default	On breach
per nested tool call (tool_timeout)	120 s	RuntimeError: tool X timed out after 120s inside the snippet
per run_code, wall clock (max_wall)	300 s	in-flight calls dropped, snippet aborted, error result

The engine's wait for tool results is bounded by tool_timeout + 30 s.

A5. Sandboxes: in-process or monty-pool

--code-sandbox auto|in-process|pool, default auto.

Sandbox	Isolation	Heap cap
pool	each REPL scope is a checkout of a monty worker subprocess (monty-pool); a crash or hang kills only the worker	--code-max-memory, default 4 GiB, enforced by monty-alloc in the worker
in-process	engine thread inside the agent	none

auto picks pool when a worker binary is found (--monty-worker, then MONTY_WORKER, then monty on PATH). Otherwise it uses in-process and logs a warn that there is no heap cap. Install the worker with cargo install monty-runtime --locked.

Both sandboxes speak the same host protocol (HostEvent::Call / Done out, call results in), so the async host loop, logging, and audit do not depend on the sandbox. A pool worker crash or timeout becomes an error result that says the REPL state was reset. The pool's request_timeout is the snippet time limit plus 10 s.

A6. Mutating-call audit

A nested call to a tool annotated destructive_hint: true or read_only_hint: false is logged at warn as mcp.tool_call / mcp.tool_result with "mutating": true. This is always on, without --verbose. The kcp read tools are annotated read-only. apply_object, patch_object, and scale_object are non-read-only and non-destructive; delete_object is destructive.

A7. Session audit

ToolCallOutput carries an audit value. For run_code it is {"nested_calls": [{tool, arguments, is_error, bytes, mutating}]}. The Claude Code harness writes it as toolUseResult on the tool_result record. That field is UI metadata: the model never sees it on resume, and record order and the parentUuid chain are unchanged.

A8. Fewer exploratory rounds

• The kcp kcp argument is optional. When exactly one kcp is discovered it is used; otherwise the error lists the ids.
• Built-in helpers are fed into every new REPL scope (and after restart): items(x), names(x), dig(obj, "a.b.c", default=None), count_by(xs, "dotted.key"). asyncio, json, and re are pre-imported; live runs showed the model forgetting import asyncio.

Live results (local 27B model, real kcp)

Prompt	Before amendments	After
socialweb kinds in root:app	KeyError, 8 rounds, no reply	2 rounds, 1 run_code, 22 506 B in → 871 B out
configmaps + events gather	4 rounds, bytes_out > bytes_in	2 rounds, 1 run_code, 6 015 B in → 77 B out
apply → patch → get → delete	4 failed applies before success	first apply succeeds; every write logged warn
6 GiB allocation	(in-process: would grow the agent)	MemoryError at 4 GiB, worker reset, agent alive

• --code-preamble PATH runs operator Python after the helpers in every new scope.
• The prompt and tool description say: plan, then write one complete run_code that fetches, filters, and returns the answer; inspect shapes only after a failure.

A9. Turn metrics

With --verbose, the backend emits model.turn_summary at the end of each run:

Field	Meaning
rounds	model requests in the run
run_code_calls	model tool calls
nested_calls	MCP calls made from Python
bytes_in	MCP result bytes received in the sandbox
bytes_out	tool-result bytes sent to the model
final_round_forced	whether the tool-free final round was used

bytes_in / bytes_out is the context saving code mode buys.

A10. REPL per conversation thread

Studying the core (ADR 0001) showed that runs, not threads, owned the REPL:

• A coalesced follow-up run (queued_run) started with an empty REPL.
• Thread history keeps only user and assistant text, so that run also lost every earlier tool result.

RunRequest now carries thread_id, and ToolExecutor::thread_scope returns one REPL per thread (LRU of 8). The runtime never overlaps runs on one thread, so the scope needs no extra locking. The run_code description tells the model that variables persist across the conversation and that earlier tool results are not in its history.

A11. Session review fixes (session e4017aa9)

A 42-minute live session (46 run_code calls, 8 errors) showed the model re-fetching data it already held, shadowing helpers, and misreading empty results.

• Variables note. After each snippet, a probe in the same scope describes the module-level names snippets bound (ss: list[80]). ToolExecutor::context_note appends [run_code variables still in scope: ...] to the newest user message of the next run. It never changes the tool description, so the cached prompt prefix stays stable. Names are found line by line (code_mode::scope); a miss only means a name is not listed. At most 30 names, 600 chars.
• Reserved helpers. A snippet that rebinds items, names, dig, count_by, or to_json at module level is rejected before it runs. Live: names = [] in one turn broke names(v) three turns later.
• dig(obj, *path, default=None). Segments may be dotted or split: dig(i, "spec.payload", "requesterDid"). A trailing non-str/int argument is still taken as the default (dig(x, "a", {})). count_by(xs, *path) likewise.
• to_json(x, indent=None). json.dumps with str() for unknown types; Monty's json.dumps has no default=.
• Hints. Known Monty gaps in a traceback add one Hint: line (json.dumps(default=), unhashable keys).
• Un-awaited awaitables (<gather(4)>) are errors, not results.
• Empty results are explicit text, not {}.

Live after the fixes: a follow-up question about data fetched the turn before made 0 MCP calls (the model used items_ from the note, avoiding the reserved items).

Amended invariants

10. A nested call reaches exactly one MCP server, or none if it fails validation.
11. Every mutating nested call produces a warn log line, regardless of flags.
12. With the pool sandbox, no Python runs in the agent process.
13. No nested call or run_code call waits forever: per-call and wall deadlines always apply.
14. Helper names are never rebound by model code; the variables note never changes the tool definitions.