A failing integration test, a dataset audit, and a review spanning several services can all involve AI agents. Their coordination needs are quite different. The bug may depend on one person holding the whole execution path in mind. The audit may repeat the same operation thousands of times. The system review may contain independent investigations whose conclusions must eventually agree.
Choosing a workflow starts with the shape of the work, its constraints, and the evidence needed to accept the result. This post walks through the coordination patterns and their costs, with illustrative engineering examples. The examples are design choices to evaluate against your own workload; they make no benchmark claims about which arrangement is fastest or cheapest.
The starting rule
Use the simplest arrangement that can meet the acceptance criteria. Add parallel work for independent tasks, verification for a quality problem, and another management layer when a subtask needs coordination of its own.
Multi-tier pyramid
The multi-tier pyramid is a hierarchical delegation workflow: a root coordinator assigns broad areas to manager agents, and each manager delegates narrower tasks to workers. Managers plan their area's work, reconcile findings, and return an integrated result. The root remains responsible for the final deliverable.
Consider a system review. A backend manager might need to reconcile API behavior, persistence rules, and migration compatibility before the root compares those findings with frontend assumptions. That intermediate integration is useful work. A manager who only forwards a question and repeats an answer adds a handoff without resolving anything.
Every layer can omit constraints, duplicate analysis, or wait on a slow branch. Put limits on depth, concurrent work, and total effort. Preserve original requirements and exact evidence references, and require unresolved disagreements to travel upward. Different branches can have different depths; the structure should follow their coordination needs.
Context separation and permissions
An agent's role description does not enforce access restrictions. Separate sessions may share a filesystem, and their initial context depends on the runtime. Check actual tool permissions, model controls, and workspace isolation before relying on them.
Workload size alone is a weak reason to add managers. A thousand independent records may suit a batch job. A small change across a tightly coupled interface may need careful coordination. The useful question is whether a branch contains decisions and dependencies that someone must actively reconcile.
One agent with tools and selective context
For an unfamiliar bug, keeping the reproduction, suspected cause, and proposed fix in one session can be valuable. Load relevant code as needed and save important discoveries. If context becomes the problem, first consider better retrieval, durable notes, and compaction. Each can omit details, so keep original evidence accessible. Additional agents introduce their own context-transfer problem.
Code-driven workflow or DAG
When the steps are known, ordinary code can enumerate inputs, schedule dependencies, and validate results. A directed acyclic graph, or DAG, represents steps that become eligible to run when their prerequisites finish. Independent branches can run together; dependent ones wait.
Checkpoints, caching, and retries must be implemented deliberately. A repeatable scheduler does not make model output deterministic, and retries need stable item identifiers and protection against duplicated side effects. Keep operations that require no model judgment in ordinary code.
Flat fan-out with one integration owner
Flat fan-out works when each worker can answer a bounded question and one owner can combine the answers. Independent branches run in parallel and return evidence to the integrator. Predetermined branches are a parallelization workflow; when an agent decides how to break down each incoming task and assigns the resulting work, the arrangement follows the orchestrator-workers pattern. Similar diagrams can therefore hide different choices about who controls the work.
Sequential pipeline
A sequential pipeline fits ordered transformations such as extracting facts, normalizing records, and drafting a report. Each stage needs an input and output contract; it does not automatically need its own autonomous agent. Dependent stages remain serial for one item, even if several items move through the pipeline at once. When successive model calls consume the previous call's output, this is prompt chaining.
Generator and verifier loop
A generator and verifier loop can improve any of these arrangements: produce an artifact, evaluate it against agreed criteria, then accept it, return specific defects, or escalate. When one model call evaluates another's output and feeds back improvements, this is the evaluator-optimizer pattern. Verification can also use objective tests and independent evidence. Two agents can make the same mistake; agreement alone does not establish correctness.
Define a review rubric before the loop begins. For a compatibility patch, that might cover the reproduced failure, behavior of existing clients, required tests, and adherence to the allowed scope. The verifier should identify actionable failures and the evidence behind them.
Also define an exit. An illustrative policy might allow two revision attempts, then return remaining failures to the integration owner for a decision. That number is a budget choice for the task, not a universal rule. Without a limit, a review loop can keep spending effort while repeating the same disagreement.
If speed or cost drives the architecture choice, compare it with a simpler baseline on representative tasks using the same acceptance criteria. Record total spend, elapsed time, output quality, and human intervention. Include coordination, retries, review, and integration. Finishing individual branches quickly is only useful if the final accepted result also improves.
Routing and peer handoffs
Routing selects a specialist for an incoming request. Peer handoffs let specialists transfer responsibility as the diagnosis changes. For example, intake might send a support request to billing, which transfers it to technical support after identifying a service failure. Record the current owner, preserve relevant findings, and bound transfers to prevent loops. Routing chooses where work goes; it does not require every specialist to run concurrently.
Workflow comparison
The table compares where each workflow fits and the costs to watch. Some patterns describe the structure of the work; others add shared state or verification to that structure.
| Workflow name | When it fits | Cost to watch |
|---|---|---|
| Multi-tier pyramid | A delegated area needs its own planning and integration | Extra latency and lost requirements at handoffs |
| One agent with tools and selective context | Discovery with tightly connected decisions | Serial work and growing context |
| Code-driven workflow or DAG | A repeated procedure with known steps | Implementation, maintenance, and safe retries |
| Flat fan-out with one integration owner | Investigations that can proceed independently | Duplicate effort and too much output to reconcile |
| Sequential pipeline | Ordered transformations of an artifact | Early errors reaching later stages |
| Generator and verifier loop | Quality is limiting acceptance | Review expense and shared blind spots |
| Shared workspace or blackboard | Specialists repeatedly need evolving information | Stale versions and competing writes |
| Routing and peer handoffs | Different request categories need different expertise | Misrouting and handoff loops |
Define the decision before choosing roles
Start with the deliverable. A patch that fixes a reproduced defect, a report whose findings link to source evidence, and a validated set of dataset labels require different checks. Write down what would make the result acceptable before deciding who should produce it.
Then identify the constraint that matters most: elapsed time, quality, cost, context size, or the effort needed to combine results. Map which steps depend on others and which files or data they share. Include practical limits such as available tools, concurrency, permissions, and where sensitive data may be processed. These facts determine whether a proposed workflow can actually run.
It helps to separate four choices that are easy to collapse into a single label like “multi-agent”:
- Control: does code determine the next step, does an agent decide, or do they share that responsibility?
- Structure: does the work run in one session, in sequence, in parallel branches, or through managers and workers?
- State: how do participants retain and exchange information: messages, saved artifacts, or structured shared records?
- Verification: what establishes that the output is acceptable: tests, source checks, human review, or a bounded revision loop?
These choices combine. A script can schedule parallel model calls, collect their artifacts, and send questionable results to a reviewer. A single agent can use tests and durable notes. A hierarchy can use the same shared records as a flat team.
Make handoffs and ownership explicit
A delegated task needs enough context to succeed without reconstructing the entire conversation. Include the original goal, relevant constraints, bounded scope, dependencies, permitted actions, acceptance criteria, and the expected return format. Keep authoritative references intact as the task passes between people or agents.
Here is an illustrative handoff for one branch of an API review:
Goal: assess whether the proposed API change preserves existing clients.
Scope: inspect the API schema and the client contract tests at the supplied revision.
Dependencies: use the interface proposal attached to the task; flag missing decisions.
Allowed actions: read files and run existing checks; do not modify implementation or tests.
Acceptance: each finding identifies an affected contract, supporting evidence, and a verification step.
Return: a concise finding list, exact source references, checks run, and unresolved questions.
Integration owner: the coordinator resolves conflicts with the service implementation review.
For a large result, save detailed evidence in an artifact and return its location and version with a short account of findings, assumptions, and validation status. This keeps the coordinator's immediate context manageable while retaining detail. Someone still has to read the evidence that matters before accepting the conclusion.
Parallel reading with one integration owner is a useful engineering starting point. Parallel editing also works when ownership is distinct, interfaces are agreed, changes are isolated, and the combined result is checked. Assign the integration responsibility before starting the workers.
Three tasks, three starting points
These examples illustrate the decision process. They are proposed arrangements, not reports of measured outcomes.
A failing integration test
Start with one agent that can reproduce the failure, follow the execution path, implement a focused correction, and run the relevant checks. The diagnosis and implementation are tightly coupled. Add a separate review if the change affects a sensitive contract or the initial validation leaves a material uncertainty. The trigger is an observed need for another check.
A recurring image dataset audit
Use code to enumerate batches and maintain stable item identifiers. Call a model where visual judgment is needed, validate the returned records, and aggregate flagged items. Independent batches can run concurrently within resource limits. Add targeted verification for uncertain cases. This combines explicit scheduling, parallel work, and quality checks without requiring a management hierarchy.
A review of twenty modules, four sharing an API
Start with flat reviewers for independent areas and one integration owner. Treat the four coupled modules as a connected review area with a common interface reference. If reconciling that area requires substantial internal planning and coordination, assign it a manager. Keep simpler branches flat.
The return should identify defects with evidence, resolve duplicate reports, and flag disagreements about the shared API. If combining the findings becomes the bottleneck, first narrow worker scopes and improve their return format. Add another coordination layer only if it has a concrete integration job to do.
Write a recommendation you can evaluate
Before implementing a workflow, capture the choice in a short recommendation. State the deliverable, dependencies, constraints, and priority, then explain how the proposed responsibilities and checks address them.
For example, a review of a service and two clients before an API migration might start with this recommendation:
Setup: one coordinator and two reviewers, with at most three agents running concurrently.
Why: the service and clients share a schema, so they need one agreed contract and a combined compatibility check. Review quality is the priority.
Responsibilities: the coordinator records the contract and resolves disagreements. One reviewer examines the service; the other traces both clients against that contract.
Checks: findings cite exact source locations. Acceptance requires the agreed compatibility checks and an explicit disposition for each finding.
Trade-off: the client review stays serial within one branch, reducing handoffs but potentially increasing elapsed time.
Change condition: if the clients prove independent and their review dominates elapsed time, give each its own reviewer while retaining the shared concurrency limit.
A useful recommendation makes one clear choice, explains the task facts or assumptions behind it, assigns responsibilities and checks, identifies the main trade-off, and gives an observable condition for changing course. Resolve missing information when it could materially alter the choice; otherwise make the assumption visible.
Keep that recommendation beside the acceptance criteria. After a representative run, compare the expected bottleneck with what actually consumed time and effort. That evidence gives the next workflow decision a firmer basis.
Further reading
Anthropic's Building effective agents discusses composable patterns including prompt chaining, parallelization, orchestrator-workers, and evaluator-optimizer loops. Its box-and-arrow explanations inspired the presentation of the original diagrams in this article.