The Orchestrator's Tools Are the Other Agents

A standard run through this layer makes six decisions: consult the strategist, assign the writer, request an editor review, request a revision, request a review again, finalize.

That sequence is written down nowhere in the code.

It is what the orchestrator model chose, one tool call at a time, given a brief and a shared state object it could read. Change the editor's verdict and the sequence changes with it. That is the layer: a crew of specialists, a structure they hand work through, and a controller that picks who acts next.

One caveat belongs here rather than buried in the limits, because it changes how you should read everything below. This layer is reachable at POST /api/v1/crew/orchestrate, and it is covered by backend/tests/test_orchestrator_agent.py, backend/tests/test_orchestrate_endpoint.py and the eval harness. No screen in the product calls it. The two paths a user actually walks are the chat, which routes by pattern and keyword matching in backend/app_new/crew/orchestrator.py, and Compose, which runs a fixed Strategist, Writer, Editor relay with the order written down in the code. What follows is the design of the layer as built and tested, not an account of what answers a brief on the live site.

Here is what the crew is. A strategist proposes angles for a brief. A writer drafts content against those angles. An editor scores each draft for brand-voice fit and flags the ones that need another pass. The orchestrator directs them. All four are the same base class talking to the same model, differing only in system prompt and output contract.

The interesting engineering sits between the four rather than inside any one of them.

The specialists exchange work through one typed object, never through each other's prose

The blackboard lives in backend/app_new/crew/state.py. A single CrewRunState dataclass carries the brief, the angles, the drafts, the reviews, a revision counter, and a transcript. Specialists read from it and write to it. No specialist ever receives another specialist's output interpolated into its prompt.

The distinction sounds pedantic until you try to debug the alternative. When agent B's prompt is an f-string containing agent A's paragraph, there is no artifact to inspect, no type to assert against, and no way to tell a routing failure from a writing failure. Both look like bad text.

Every write goes through a named method, and each one records a turn:

    def record_reviews(self, reviews: list[Review], summary: str) -> None:
        self.reviews = reviews
        self.phase = Phase.REVIEW
        self.transcript.append(Turn("editor", "review", summary))

    def record_revision_request(self, summary: str) -> None:
        self.revision_count += 1
        self.phase = Phase.REVISION
        self.transcript.append(Turn("orchestrator", "request_revision", summary))

No caller sets self.reviews directly, which means no caller can update the reviews without also advancing the phase and appending to the transcript. The failure mode is a half-updated state: reviews present, phase still DRAFTING, transcript missing the turn, and a run that reads as though the editor never spoke. Bundling the three writes into one method makes that state unreachable rather than merely unlikely.

The transcript is not a log. It is persisted with the finished draft, and it is what the SSE stream renders so a human watching a run sees the crew deciding rather than a spinner.

The orchestrator is an agent, and its tools are the specialists

In backend/app_new/crew/orchestrator_agent.py, the controller has exactly five tools:

def _delegation_tool(name: str, description: str) -> Dict[str, Any]:
    return {
        "name": name,
        "description": description,
        "input_schema": {"type": "object", "properties": {}},
    }


DELEGATION_TOOLS = [
    _delegation_tool("consult_strategist", "Ask the Strategist to propose angles for the brief."),
    _delegation_tool("assign_writer", "Ask the Writer to draft content from the chosen angles."),
    _delegation_tool("request_editor_review", "Ask the Editor to score the drafts and flag revisions."),
    _delegation_tool("request_revision", "Send the work back to the Writer using the Editor's feedback."),
    _delegation_tool("finalize", "Finish the run and return the best draft."),
]

Look at the input schemas. They are empty objects. The orchestrator chooses who acts, never what they act on. The specialist reads its context out of shared state.

That was deliberate, and it is the design decision I would defend hardest. The moment a delegation tool takes arguments, the orchestrator model starts re-typing the brief into them, paraphrasing as it goes, and you are back to string handoffs with a JSON schema wrapped around them. Empty inputs make the blackboard the only channel.

The run loop asks the model for one tool call, executes it, feeds a compact JSON result back, and asks again. There is no if statement choosing the order. When the writer finishes, the loop does not know that review comes next. It hands back {"phase": "drafts_done", "draft_count": 1} and the model decides.

The loop also carries a step budget, derived from the revision cap:

        if max_steps is None:
            # base flow (4) + two steps per possible revision + slack
            max_steps = 6 + 2 * state.max_revisions

It is a runaway backstop, sized so a run can always reach finalize on its own. If the loop ever exits on the budget instead of on a decision, that is logged as step_budget_exhausted, because a silent finish there is indistinguishable from a clean run.

Every specialist answers in a schema, because prose has to be parsed and schemas do not

Each specialist is pinned to exactly one tool. From backend/app_new/crew/base_agent.py:

            response = await self.client.messages.create(
                model=self.model,
                max_tokens=max_tokens or settings.ANTHROPIC_MAX_TOKENS,
                system=system,
                messages=messages,
                tools=[tool],
                tool_choice={"type": "tool", "name": tool["name"]},
            )

One tool in the list, tool_choice pinned to it. The model cannot answer in paragraphs. The strategist returns {"angles": [...]}, the writer returns {"drafts": [...]}, the editor returns {"reviews": [...]}, and each maps directly onto a frozen dataclass in state.py.

When the model answers with something other than that tool call, the code does not shrug and continue:

        self._record_run(
            api_kind="forced_tool",
            status="error",
            latency_ms=latency_ms,
            usage=usage,
            stop_reason=response.stop_reason,
            error_class="schema_mismatch",
        )
        raise ValueError(f"{self.AGENT_NAME} did not return the {tool['name']} tool call")

Two things happen there and both matter. It raises, so nothing empty gets persisted. And it writes a telemetry row classified as schema_mismatch, distinct from a provider error, so contract drift after a model upgrade shows up as a number rather than as vague reports of bad output.

One field in the editor's schema earns a story. The review carries draft_index, an integer, and draft_title is marked human-readable only. It used to select by title, matching the editor's model-generated string against the writer's model-generated string. Any paraphrase broke equality and the fallback grabbed the first draft. Reproduced against the real code, a run scoring 55 and 95 returned the 55.

The failure mode is a model-generated string used as a join key. The model has one job it can reliably do here, which is emit the index it is already reasoning about. An index that does not resolve is discarded rather than quietly collapsing onto the first draft:

    def best_draft(self) -> Optional[Draft]:
        if not self.drafts:
            return None
        scored = [r for r in self.reviews if 0 <= r.draft_index < len(self.drafts)]
        if not scored:
            return self.drafts[-1]
        return self.drafts[max(scored, key=lambda r: r.brand_voice_score).draft_index]

The dispatcher refuses illegal moves and tells the model why

max_revisions started life as a sentence in the orchestrator's system prompt. Then it got measured against a model that kept asking for one more pass. Ten revisions against a cap of two.

A prompt is a request. Requests get declined.

So _dispatch became the enforcement point. Before any delegation executes, _refusal checks the move against actual state: a revision past the cap, writing with no angles on the board, reviewing with no drafts. Illegal moves never run:

        refusal = self._refusal(name, state)
        if refusal is not None:
            self.logger.info("delegation_refused", tool=name, reason=refusal)
            state.transcript.append(
                Turn("orchestrator", f"refused:{name}", refusal)
            )
            return {"refused": refusal, "phase": state.phase.value}

        return await handler(state)

A refusal comes back as a return value rather than an exception, shaped like every other tool result, and it goes back into the conversation. The refusal text names the move to make instead: the one for writing without a strategy says there are no angles yet and points at consult_strategist. The model reads that on its next turn and corrects itself.

The model chooses among legal moves. Deciding which moves are legal was never the model's job.

That is what keeps this agentic rather than a pipeline with a model bolted on. Nothing hardcodes the order. The state machine only removes options that cannot be coherent, and it explains each removal in the same channel the model is already reading.

An eval drives this with a rogue policy that asks for request_revision forever. The cap holds at two, and the transcript carries the refusals as first-class turns.

The revision loop is the editor's verdict, not a counter

Evaluator-optimizer is the pattern, and it is three lines of return value. After the editor scores, the dispatcher hands back:

        return {
            "phase": "review_done",
            "needs_revision": state.any_review_needs_revision,
            "revisions_exhausted": state.revisions_exhausted,
        }

That is the entire loop condition, and the orchestrator model, not the code, acts on it. request_revision bumps the counter, records the turn, and re-runs the writer with the editor's feedback appended to the prompt. Then review happens again.

The evals pin the three shapes this can take:

standard_brief_one_revision       positive     1    6   10  PASS
editor_passes_no_revision         positive     0    4    6  PASS
editor_never_satisfied_bounded    edge         2    8   14  PASS

One revision when the editor flags the first draft. Zero when the editor is satisfied on the first pass, which is the case that proves the flow is routing rather than a fixed script. Two, exactly the cap, when the editor is never satisfied.

Request-scoped state travels as call arguments, never on the instance

The orchestrator holds one strategist, one writer, and one editor for the whole run, so their prompt caches survive across delegations. Shared instances plus per-request state is a concurrency bug waiting for its second user, so per-call context goes through a kwarg:

        system = await self.build_system_prompt(query=query, **(system_extras or {}))

The writer consumes it in backend/app_new/crew/writer.py, where an executive profile becomes its own cached system block:

        ghost_text = self._build_ghostwriting_section(ghostwrite_exec)
        if ghost_text:
            blocks.append({
                "type": "text",
                "text": ghost_text,
                "cache_control": {"type": "ephemeral"},
            })

Nothing is assigned to self. The failure mode is invisible until load: two runs in flight, one overwrites the other's executive, and someone's CEO post is written in someone else's voice.

Cache-block ordering is doing real work in that method too. The stable prefix carries the breakpoints, and the volatile per-query retrieval is appended last, after every breakpoint, so a fresh retrieval never invalidates the cached prefix above it.

Where this layer stops

Prompts are not a control surface, and the sharpest proof of that came from outside the orchestrator. Walking the product on an org with zero executive profiles on file, the crew assigned quotes to "Sarah Chen, CEO" and the writer signed a draft "By [CTO Name]". Both were invented whole.

The fix was not a better instruction. The roster goes into the prompt, and any executive name that does not resolve to a stored profile is dropped server-side before persistence. Bracketed placeholders are detected and surfaced as a warning rather than stripped, because a warning tells the human something is missing and a silent strip just leaves a smoother lie.

Now the honest limits.

The first limit is reach. Grep the web app for the orchestrate endpoint and there is no call to it. The delegation layer is real code with real tests and a real eval suite, and it is still not what runs when someone briefs the crew today. Everything below is a limit of a component sitting one wire short of the product.

The eval suite is ten cases and it passes ten out of ten, which sounds better than it is. Seven of those cases route through a fixed if/elif in backend/app_new/crew/mock_provider.py that returns assign_writer after strategy_done and request_editor_review after drafts_done. Three stub the decision function outright: one that spams revisions, one that skips steps, and one that fails at the decision step. Zero of the ten call a live model. The suite proves the enforcement holds against a controller I wrote to misbehave in ways I anticipated. It says nothing about a controller misbehaving in a way I did not.

Beyond that: the crew ships four specialists and the orchestrator instantiates three, because the analyst is not one of its tools. CrewRunState carries exec_name, and the /orchestrate endpoint fills it from the request, but the orchestrator agent never reads it, so ghostwriting only works on the older fixed /compose path. And the empty delegation schemas that keep handoffs clean also mean the orchestrator cannot say "draft this shorter, for a skeptical audience". It can only say assign_writer. Every instruction beyond that has to reach the specialist through state or through its system prompt, and adding a new kind of instruction means adding a field, not a sentence.

That last one is a real cost. I would still take it. A layer where handoffs cannot silently become string interpolation is worth more than a layer where the orchestrator can improvise briefs.

The one-liner: an orchestrator that cannot be talked out of its own invariants is the difference between a crew and a costume.

Next in the series: the memory layer, and a row that looks embedded everywhere a human might check.

The views expressed here are my own and are not related to or reflective of my work or any organization I am affiliated with.