We have been developing a bridge tool between Mathematica and Claude Code for some time (see this article and GitHub site).
In this system, a function called ClaudeEval generates Wolfram Language code from prompts and automatically executes it, with the goal of interactively integrating Claude Code within Mathematica notebooks.
LLM calls are represented as LLMGraph (currently a custom data structure, though we would like to integrate it with LLMGraph if possible), and the system includes an asynchronous execution scheduler that runs them.
LLM calls are represented as LLMGraph (currently a custom data structure, though we would like to integrate it with LLMGraph if possible), and the system includes an asynchronous execution scheduler that runs them.
In[]:=
$Language="English";
In[]:=
ClaudeEval["The sum from 1 to 100"]
In[]:=
Sum[i,{i,1,100}]
Out[]=
5050
Done. Check the code output, then use to continue.
The key point is that information from cells and variables that the user needs to keep confidential is not sent to the cloud LLM; only schema information is passed, allowing expression construction while preserving the confidentiality of the processed data. If a local LLM is specified, confidential data can be processed, but the envisioned usage is to process only confidential cells with $ClaudePrivateModel, process the remaining cells with $ClaudeModel, and then merge the results.
In[]:=
{$ClaudeModel,$ClaudePrivateModel}
Out[]=
{claude-opus-4-6,{lmstudio,qwen/qwen3.5-35b-a3b,http://192.168.2.110:1234}}
In[]:=
ClaudeEval["The sum from 1 to 100",Model->$ClaudePrivateModel]
✓ lmstudio/qwen/qwen3.5-35b-a3b response obtained.
In[]:=
sum=Total[Range[1,100]];Print["Sum from 1 to 100: ",sum]
Sum from 1 to 100: 5050
For this reason, we created a layer for referencing notebook cells (NBAccess) and ensured safety by centralizing all cell read/write operations through its API. (Incidentally, the "safety" here refers to preventing accidental user mistakes; mechanisms for so-called security protection could be added but are not currently in place.)
Even when expressions are generated by ClaudeEval, errors can occur. In such cases, retries are allowed up to a maximum number of iterations. However, when potentially dangerous code was generated, only minimal checking mechanisms were in place. As a result, the only countermeasure was a passive one: defining an additional function called ClaudeQuery that does not execute automatically, and using it for everyday LLM calls to prevent accidental user mistakes.
Furthermore, since ClaudeEval permits nested generation, it is possible to call ClaudeEval recursively to achieve pseudo-iterative loop execution. However, as described above, the code verification mechanisms were too weak, making it difficult to scale this approach up.
Even when expressions are generated by ClaudeEval, errors can occur. In such cases, retries are allowed up to a maximum number of iterations. However, when potentially dangerous code was generated, only minimal checking mechanisms were in place. As a result, the only countermeasure was a passive one: defining an additional function called ClaudeQuery that does not execute automatically, and using it for everyday LLM calls to prevent accidental user mistakes.
Furthermore, since ClaudeEval permits nested generation, it is possible to call ClaudeEval recursively to achieve pseudo-iterative loop execution. However, as described above, the code verification mechanisms were too weak, making it difficult to scale this approach up.
However, with the Claude Code leak on 3/31, techniques for implementing iterative-loop agent architectures became shared knowledge overnight. Here, referencing the implementation of Claw-Code (https://github.com/ultraworkers/claw-code), we extended our system to support iterative-loop agent implementation.
Architecture
Architecture
ClaudeRuntime.wl and ClaudeTestKit.wl are the files added in this update. The dependency principles are as follows. By adding ClaudeRuntime to the existing NBAccess/claudecode, a new ClaudeEval was constructed in ClaudeRuntime in parallel with the legacy ClaudeEval in claudecode.
At this scale, a mechanism for running tests becomes indispensable. Claw Code likewise implements tests on the Python side that do not actually invoke the LLM, while production execution is carried out on the Rust side. ClaudeTestKit is what handles that testing mechanism and other verification tasks.
ClaudeRuntime
ClaudeRuntime
The core of ClaudeRuntime is the Expression-Proposal loop, where each turn consists of the following phases.
Whereas a typical agent runtime centers on tool calls, in this system the LLM does not directly touch resources but instead proposes Held Mathematica expressions.
1
.BuildContext — calls the adapter's BuildContext function to construct the context packet.
claudecode has NBAccess create a safe context packet
2
.QueryProvider — Querying the LLM provider. Supports both asynchronous mode (via claudecode's LLMGraph DAG) and synchronous mode (for testing and local LLMs).
3
.ParseProposal — Parses the LLM's response and structures it as either a Mathematica expression (HoldComplete[...]) or a text response.
4
.ValidateProposal — Determines safety by referencing NBAccess's `$NBDenyHeads` / `$NBApprovalHeads`.
5
.DispatchDecision — Branches into execution, awaiting approval, or a repair turn depending on the validation result (Permit / NeedsApproval / Deny / RepairNeeded / TextOnly / ToolUse).
However, the safety judgement itself is performed by NBAccess.
Turn execution is deployed as a DAG job using LLMGraphDAGCreate from claudecode (https://github.com/transreal/claudecode ). Each phase is defined as a DAG node and executed sequentially according to its dependencies. In asynchronous mode, the queryProvider node launches a CLI process and the collectProvider node writes the results back to the RuntimeState, thereby handling LLM calls in a non-blocking manner.
ClaudeTestKit
ClaudeTestKit
ClaudeTestKit is a test harness designed for systematically testing the runtime and security kernel of ClaudeRuntime (https://github.com/transreal/ClaudeRuntime ).
Testing ClaudeRuntime presents a fundamental dilemma: if everything is replaced with mocks, it becomes impossible to verify the behavior of the most critical component—the security kernel itself. ClaudeTestKit addresses this problem by adopting the design principle of "limiting mocks to the LLM provider and adapter interface layers, while testing the core of the security kernel as much as possible through the real NBAccess (https://github.com/transreal/NBAccess ) public API."
Testing ClaudeRuntime presents a fundamental dilemma: if everything is replaced with mocks, it becomes impossible to verify the behavior of the most critical component—the security kernel itself. ClaudeTestKit addresses this problem by adopting the design principle of "limiting mocks to the LLM provider and adapter interface layers, while testing the core of the security kernel as much as possible through the real NBAccess (https://github.com/transreal/NBAccess ) public API."
◼
MockProvider: A fake Provider that eliminates LLM uncertainty so that only the Runtime's behavior can be tested
◼
MockAdapter: A fake implementation of the adapter interface required by ClaudeRuntime (its role is to test the Runtime's state transitions without using NBAccess, real Notebooks, or real APIs)
◼
scenario runner: The executor for Runtime tests (receives test scenarios, creates a Runtime / passes input and executes turns / waits until completion, failure, or pending approval / retrieves RuntimeState and EventTrace / runs assertion groups)
◼
Golden normalization: enables mechanical comparison between the previous correct trace and the current trace. Normalization for so-called golden tests.
◼
no-leak assertion: stringify the entire trace and verify that it does not contain any specified secret strings
◼
validation-denied assertion: Tests whether the Runtime correctly rejected a prohibited operation
◼
For example, with MockAdapter, heads such as
DeleteFile[“important.txt”]
RunProcess[...]
Run[...]
are denied.
DeleteFile[“important.txt”]
RunProcess[...]
Run[...]
are denied.
◼
update transaction regression tests: Regression tests safeguarding the safety of automatic package updates
◼
Transactions such as package updates are more dangerous than ordinary one-shot execution.
Therefore, the following phases are reproduced using mocks:
SnapshotPackage/ApplyToShadow/StaticCheck/ReloadCheck/RunTests/CommitTransaction/RollbackTransaction
Then, by specifying something like FailAtPhase -> "StaticCheck", a failure is triggered at a specific phase.
Therefore, the following phases are reproduced using mocks:
SnapshotPackage/ApplyToShadow/StaticCheck/ReloadCheck/RunTests/CommitTransaction/RollbackTransaction
Then, by specifying something like FailAtPhase -> "StaticCheck", a failure is triggered at a specific phase.
◼
The purpose is to confirm that: commits are not made on failure / rollbacks are performed when necessary / execution does not proceed when a snapshot has not been taken / test failures are not treated as successes / transaction bugs that were previously fixed do not regress.
The Concept of Recursion
The Concept of Recursion
In the current ClaudeEval, "recursion" primarily refers to proposal continuation, format retry, and validation repair (in that sense, it has not yet reached the stage of true agentic behavior).
In the conventional ClaudeEval, a single recursion call limit was used to collectively control infinite loop prevention, retry caps on errors, and termination of multi-step execution. However, this meant that distinct types of events were tracked under the same counter. In the new design, recursion is expressed not as "depth of self-invocation" but as state transitions and budget consumption.
Specifically, the recursion limit is decomposed into per-category budgets and defined as a RetryPolicy:
<|"Profile"->"Eval"|"UpdatePackage","Limits"-><|"MaxTotalSteps"->6,"MaxProposalIterations"->3,"MaxTransportRetries"->2,"MaxFormatRetries"->2,"MaxValidationRepairs"->1,"MaxExecutionRetries"->0,"MaxReloadRepairs"->0,"MaxTestRepairs"->0,"MaxPatchApplyRetries"->0,"MaxFullReplans"->0|>,"Backoff"-><|...|>,"ClassificationRules"-><|...|>,"CheckpointPolicy"-><|...|>,"ApprovalPolicy"-><|...|>,"Accounting"-><|...|>|>
RuntimeState is a dynamic state that represents "how far execution has progressed and how many times each resource has been consumed."
例:
<|"RuntimeId"->"...","Profile"->"Eval"|"UpdatePackage","Status"->"Initialized"|"Running"|"AwaitingApproval"|"Retrying"|"Done"|"Failed","CurrentPhase"->...,"RetryPolicy"-><|...|>,"BudgetsUsed"-><|...|>,"LastContextPacket"->...,"LastProposal"->...,"LastValidationResult"->...,"LastExecutionResult"->...,"LastFailure"->...,"FailureHistory"->{...},"EventTrace"->{...},"CheckpointStack"->{...},"PendingApproval"->...,"ConversationState"-><||>,"TransactionState"-><||>,"Metadata"-><||>|>
Examples
Examples
What follows are execution examples of ClaudeEval in ClaudeRuntime.
In[]:=
<<ClaudeRuntime`
ClaudeRuntime package loaded. (v2026-04-20T11-abort-runtime-dags)
$UseClaudeRuntime = True
CreateClaudeRuntime[adapter] Create runtimeId ClaudeRunTurn[runtimeId, input] Launch DAG jobId ClaudeContinueTurn[runtimeId] Continue turn ClaudeRuntimeState[runtimeId] Query state (lightweight) ClaudeRuntimeStateFull[runtimeId] Query state (complete, heavy) ClaudeTurnTrace[runtimeId] Event trace ClaudeApproveProposal[runtimeId] Approve ClaudeDenyProposal[runtimeId] Deny ClaudeRuntimeCancel[runtimeId] Cancel ClaudeRuntimeRetry[runtimeId] Retry failed nodes ClaudeRetryPolicy[profile] Get RetryPolicy ClaudeClassifyFailure[failure] Classify failure (for task decomposition / multi-agent orchestration, load ClaudeOrchestrator.wl)
The usage is nearly identical to the conventional ClaudeEval: it converts a prompt into an expression and executes it.
Here is the code for visualizing projectile motion thrown upward, with key physics annotations.
Dangerous code is automatically blocked from execution.
❓ Approval required: CoreContextOverwrite — Code overwrites core package functions (NBAccess/ClaudeCode/ClaudeRuntime/ClaudeTestKit). This may break system functionality. Approval required.
NeedsApproval
HoldComplete[ClaudeRuntime`Private`$iClaudeRuntimes = {}]
⛔ User denied the proposed expression.
The user stopped execution, so the Status is Failed.
A function to plot the LLMGraph has been created, but it only displays the graph and no particular functionality has been added yet. Specifying "Detail" -> True displays nodes corresponding to each phase.
Below are the details of the LLMGraph. The two on the right correspond to single-mode ClaudeEval calls from claudeeval.wl, and the remaining two correspond to calls after loading ClaudeRuntime.wl. The vertically elongated column of black nodes corresponds to buildContext→queryProvider→collectProvider→parseProposal→validateProposal→dispatchDecision. The leftmost subgraph generates three LLM queries corresponding to "Graph of projectile motion thrown upward (All explanations shouled be in English).", but since turn1 and turn2 required repair, retries were repeated, and the final turn3 is the one that was output.
Because the LLMGraph and scheduler are separated, if the queryProvider fails due to a network disconnection or Claude usage limits, you can retry directly with ClaudeRuntimeRetry[$ClaudeLastRuntimeId].
You can also save directly to disk with ClaudeRuntimeSnapshot[$ClaudeLastRuntimeId], then load and re-execute with ClaudeRuntimeRestore[“C:/.../snapshots/...”, “Resume”].
You can also save directly to disk with ClaudeRuntimeSnapshot[$ClaudeLastRuntimeId], then load and re-execute with ClaudeRuntimeRestore[“C:/.../snapshots/...”, “Resume”].
ClaudeOrchestrator
ClaudeOrchestrator
ClaudeEval based on ClaudeRuntime is an automaton for LLM calls that incorporates decision-making such as automatic retries, but on its own it cannot execute multiple agent tasks.
Therefore, we introduced a framework that, when a complex prompt is input to ClaudeEval, decomposes it into subtasks and launches each as an independent LLM call. When ClaudeOrchestrator is loaded, ClaudeEval decomposes prompts into subtasks according to the prompt and calls the LLM accordingly.
Therefore, we introduced a framework that, when a complex prompt is input to ClaudeEval, decomposes it into subtasks and launches each as an independent LLM call. When ClaudeOrchestrator is loaded, ClaudeEval decomposes prompts into subtasks according to the prompt and calls the LLM accordingly.
However, this has not yet been rigorously defined, including its implementation specifications, and is an experimental introduction. This part also draws on Claw Code as a reference, but it is still rough around the edges and the granularity of task decomposition is coarse. Moreover, because the Mathematica kernel is single-threaded, asynchronization is extremely difficult, and since it has not yet been properly asynchronized, it easily blocks the kernel (of course, this is an issue that should be resolved with the highest priority).
Under $ClaudeEvalMode = "Auto", orchestration (the LLM planner) is triggered when any of the following conditions are met:
- The input string is 500 characters or longer
- It contains 3 or more line breaks (a complex multi-line task)
- The input string is 500 characters or longer
- It contains 3 or more line breaks (a complex multi-line task)
Additionally, the LLM planner decomposes the task into 3 or more subtasks (default threshold = 3). This allows a single prompt to drive a 3–4 stage flow of "preliminary research → structuring → deliverable." The workers share the roles of Explore/Draft/Verify/Reduce.
Example
Example
This code scans all .wl files under $packageDirectory, extracts Needs/Get/BeginPackage references, builds the dependency graph, highlights foundational packages, detects cycles, and reports the longest dependency chains.
The code reads every .wl file, uses regex to extract Needs["..."], Get["..."], << ..., and BeginPackage` dependency declarations. The graph uses blue for foundational packages (claudecode, NBAccess), magenta for ClaudeRuntime as the new foundational candidate, and gray for everything else. Circular dependencies are listed in red, and the 5 longest transitive chains are displayed below the graph. (Double click the cell bracket to show the code)
The code reads every .wl file, uses regex to extract Needs["..."], Get["..."], << ..., and BeginPackage` dependency declarations. The graph uses blue for foundational packages (claudecode, NBAccess), magenta for ClaudeRuntime as the new foundational candidate, and gray for everything else. Circular dependencies are listed in red, and the 5 longest transitive chains are displayed below the graph. (Double click the cell bracket to show the code)
The LLMGraph generated by ClaudeOrchestrator is not integrated with the notebook history. Also, its behavior when calling local LLMs is not yet stable. Once local LLM calls become stable, agent-based work like Claude Code will be executable even in offline Mathematica. These are tasks for the future.
Summary and Future Work
Summary and Future Work
The implemented ClaudeEval function has a three-tier execution form. The first claudecode-based ClaudeEval is lightweight but has no means of handling errors or abnormal states. The ClaudeRuntime-based ClaudeEval launches an automaton for LLM calls as an independent unit. The final ClaudeOrchestrator-based ClaudeEval realizes agent-execution-type LLM calls.
claudecode.wl, together with NBAccess.wl, contains numerous tools that support Mathematica notebook-based code development and documentation. As a result, it has become an extremely large file. It is necessary to split claudecode.wl into claudecode.wl and claudecode_base.wl, and to clarify the dependency relationships as ClaudeOrchestrator→ClaudeRuntime→{claudecode_base, NBAccess}.
The framework for generating code files is handled by ClaudeCreatePackage/ClaudeUpdatePackage, but it has not yet reached a practical level. The framework for ClaudeOrchestrator-based code generation and verification needs to be redesigned and reimplemented.
Acknowledgments
Acknowledgments
Part of this software was developed with support from the Satake Technical Foundation, 2025 Research Project "Development of a Presentation Tool to Support Cognitive Decline."
*Note: This document was translated from Japanese by documentation.wl (https://github.com/transreal/documentation). See also this article.
CITE THIS NOTEBOOK
CITE THIS NOTEBOOK
Implementation of an iterative-loop agent architecture for the Wolfram - Claude Code bridge
by Katsunobu Imai
Wolfram Community, STAFF PICKS, April 14, 2026
https://community.wolfram.com/groups/-/m/t/3686091
by Katsunobu Imai
Wolfram Community, STAFF PICKS, April 14, 2026
https://community.wolfram.com/groups/-/m/t/3686091