The Journey of a Tool Call
The model outputs a tool_call — say Edit("foo.py", ...) — and the result ends up written into history. What happens in between?
This post walks that path end-to-end, disassembling two core modules: ToolOrchestrator (the dispatch layer, managing concurrency and result budgets) and ToolExecutor (the execution layer, managing permissions, optimistic locks, and circuit breakers).
The Conclusion First
The tool execution pipeline has two layers with completely separate responsibilities:
| Layer | Module | Responsible for |
|---|---|---|
| Dispatch | ToolOrchestrator | Concurrent grouping, ordering guarantees, result budget truncation |
| Execution | ToolExecutor | Permission check → optimistic lock injection → circuit breaker check → tool.run() |
The model produces a batch of tool_calls; Orchestrator handles "how to run this batch"; Executor handles "how to execute one tool safely." There's a clean interface boundary between the two layers.
1. Dispatch Layer: Concurrent Grouping, Ordering Guarantees
# tools/orchestrator.py ToolOrchestrator
SAFE_TOOL_NAMES = {"Read", "Grep", "Glob"} # read-only, safe to parallelize
UNSAFE_TOOL_NAMES = {"Edit", "Bash", "Task", ...} # side-effecting, must be serialThe model may request multiple tools in a single step — for example, Read three files and then Edit one. The Orchestrator's first job is batching:
# partition_tool_calls()
# input: [Read, Read, Edit, Grep, Grep, Edit, Read]
# output: [concurrent(Read,Read), serial(Edit), concurrent(Grep,Grep), serial(Edit), concurrent(Read)]The rule is simple: consecutive safe tools are merged into one concurrent batch; a write tool breaks the batch.
Concurrent batches run via ThreadPoolExecutor, but there's a key detail — result ordering is preserved:
def _run_batch_concurrently(self, batch, ...):
# submit returns a Future immediately without blocking — all tools start nearly simultaneously
futures = {
offset: executor.submit(self._execute_plan, plan, ...)
for offset, plan in enumerate(batch.calls)
}
# .result() blocks until done; results stored by offset, preserving original position
for offset, future in futures.items():
observations[offset] = future.result()
# rebuild list in offset order — thread completion order is non-deterministic, this forces the original
return [observations[idx] for idx in range(len(batch.calls))]Thread B may finish before thread A, but in the returned list A always precedes B. The order the model requested is the order written into history.
Serial batches are a plain for-loop: one completes before the next begins. Total time is the sum of all tool times, but there's no race condition.
2. Execution Layer: Four-Gate Pipeline
Each tool's actual execution goes through ToolExecutor.execute(), which is a linear pipeline — any gate failure short-circuits immediately:
argument parsing → [Gate 1] permission check → [Gate 2] optimistic lock injection → [Gate 3] circuit breaker check → tool.run()Gate 1: Permission Check
# tools/permissions.py RiskClassifier
# Decision priority:
# 1. Read/Grep/Glob → ALLOW (read-only, no risk)
# 2. Edit → check runtime_mode (read-only sub-agent → DENY)
# 3. Bash → regex blacklist → greylist → whitelist
# 4. unknown tool → DENY (fail-closed)Bash is the most complex case. Blacklist hits are immediate DENY without asking the user:
_BASH_DENY_PATTERNS = (
(re.compile(r"sudo"), "sudo crosses the process privilege boundary"),
(re.compile(r"rm(?:\s|$)"), "destructive delete command"),
(re.compile(r"bash\s+-c"), "nested shell execution bypasses command classification"),
(re.compile(r"`|\$\("), "shell command substitution executes nested commands"),
...
)Greylisted commands (mv, pip install, chmod, etc.) go through the ASK policy, which in the current MVP implementation defaults to DENY when ask_policy="deny" — meaning unknown-risk commands are not executed by default, forcing the model to find another approach.
The critical design principle: fail-closed. Being in the tool Registry only means the model can "see" the tool. Execution rights still have to pass through the permission gate.
Gate 2: Optimistic Lock Injection (Edit only)
# tools/executor.py
if name == "Edit":
parameters = self.registry.inject_optimistic_lock_params(name, parameters)After a Read tool executes, the framework caches the file's mtime + size. Before an Edit executes, the framework automatically injects the cached expected_mtime_ms into the parameters. If the file was externally modified between the Read and the Edit, the Edit detects the mtime mismatch and returns a CONFLICT error instead of silently overwriting.
This solves a subtle problem: the model reads a file and decides to modify it, but the file may have been changed by the user or another tool in the interim. The optimistic lock makes this class of "write clobber" detectable.
Gate 3: Circuit Breaker Check
# tools/circuit_breaker.py
# Three states: CLOSED (normal) → OPEN (disabled) → HALF_OPEN (probe after cooldown)
if not self.registry.is_available(name):
return self.registry.create_circuit_open_result(name, parameters)After 3 consecutive failures (default threshold), the circuit breaker opens and the tool is temporarily disabled for 300 seconds. This prevents a broken tool from retrying repeatedly, consuming tokens and step budget. After the cooldown period, the breaker enters HALF_OPEN — it lets one probe through: success restores CLOSED, failure resets the timer and continues OPEN.
tool.run() and the Exception Safety Net
try:
result = tool.run(parameters)
if not isinstance(result, ToolResult):
raise TypeError(...)
except Exception as exc:
# all uncaught exceptions are caught here, converted to EXECUTION_ERROR ToolResult
# guarantees no exception propagates upward; loop always receives a ToolResult
# a single tool crash cannot interrupt the entire agent
return ToolResult(status=ERROR, error_code=EXECUTION_ERROR, ...)Any internal exception from the tool is caught here and converted into a standard ToolResult. This is the tool pipeline's last safety net: a single tool crashing doesn't crash the whole loop.
3. Result Post-Processing: Two-Layer Byte Budget
After a tool executes, its result passes through three post-processing steps:
execution result
→ _normalize_empty_result() empty output gets a placeholder text
→ _apply_observation_limit() initial truncation by line/byte count
→ _apply_result_budget() two-layer byte budget final truncationTwo-layer budget prevents tool output from overflowing the context window:
Layer 1 (single-tool cap, default 50KB):
single tool output > 50KB → force_truncate → full content spills to disk file
result includes the file path; model can reference it on demand
Layer 2 (batch total cap, default 200KB):
total after per-tool truncation still > 200KB → sort by size descending,
force-truncate one by one until total is under the limit
greedy strategy: truncate the largest first to minimize truncation countResults already truncated by Layer 1 are marked replaced=True in metadata; Layer 2 skips them to avoid double-truncating the same result.
4. Lifecycle Events: Full Observability
Every tool call passes through four lifecycle states, all emitted as events to the trace/transcript:
requested → started → completed / failedrequested: model requested the tool (recorded before argument parsing, regardless of outcome)started: passed permission check, entering actual executioncompleted: execution succeeded (including partial status)failed: execution failed (including permission denial, circuit break, exception)
These four states let the trace fully reconstruct "the story of a tool call": was it requested? Was it denied or did it actually run? How long did it take? What was the result?
Design Highlights
1. Dispatch and execution are separated
Orchestrator doesn't care how a single tool executes — only "how to schedule this batch." Executor doesn't care how many tools are concurrent — only "is this one tool safe." Separation of responsibilities makes both layers independently testable and evolvable.
2. Writes are forced serial; order matches model request
After concurrent execution, results are force-reordered by original offset — this isn't just an ordering concern, it's a semantic one. When the model issues [Edit A, Edit B], it expects A before B. Wrong write order in history will confuse the model on the next step.
3. Multiple safety boundaries
Permission blacklist (rule layer) → optimistic lock (data layer) → circuit breaker (availability layer) → exception safety net (stability layer). Each layer solves one class of problem, with no overlap.
Summary
| Design choice | Approach | Engineering value |
|---|---|---|
| Concurrency strategy | Read-only concurrent, writes serial | Safe, no race conditions; Read/Grep/Glob concurrency improves throughput |
| Ordering guarantee | Offset-based reordering | Model semantics not scrambled by concurrent execution |
| Permission design | Fail-closed + regex blacklist | Dangerous commands never reach tool.run() |
| Optimistic lock | Read caches mtime, Edit auto-injects | Write clobber is detectable, not silent |
| Circuit breaker | Three-state + cooldown | Broken tools don't retry and drain step budget |
| Byte budget | Single-tool + batch total, two layers | Tool output can't overflow the context window |
About the Source Code
All analysis in this series is based on the open-source project MyCodeAgent.
The source code includes comments at key locations aligned with the walkthrough in each post — you can read the articles alongside the code, or clone it directly to run, modify, and extend, building your own agent on top of it.
git clone https://github.com/chendongqi/MyCodeAgent
cd MyCodeAgent
cp .env.example .env # fill in your LLM API key
uv sync
uv run python main.pyExplore PrimeSkills — a curated marketplace of AI agents and skills, each validated against real enterprise workflows. No hype, just what actually works.
Visit my personal site for more insights and interesting products.