Agentic Workflow Orchestrator

Overview

The orchestrator turns a goal into an explicit, inspectable plan and then executes it one step at a time. A tool registry defines the only tools the agent may call. A planner proposes an ordered plan that uses only those tools. An executor runs each step, resolving references to prior outputs, validating each output against its schema, and enforcing a step budget and retry policy. A step marked consequential pauses for human approval before it runs. Every step and every guardrail decision is written to an append-only audit trace, so a completed or halted run can be reconstructed afterwards. Swap the tools and the same core drives a different job.

The design choice the project is built around is where the model’s authority ends. The planner may propose a plan, but it has no authority over what actually runs: the executor only calls allow-listed tools, only within a step budget, and never runs a consequential action without a human’s approval. The interesting engineering is not the planning. It is the control flow that contains it.

The problem

An agent that can call tools on a user’s behalf is only as trustworthy as the limits it will not cross. The failure modes are specific and known: it loops forever, it calls a tool it was never given, it runs an irreversible action before anyone can stop it, or it passes one tool’s malformed output into the next and corrupts the run. A demo that ignores these looks impressive and is unshippable.

So the work is not making an agent act — a single model call does that. It is making an agent act inside bounds a human and a reviewer can see, and being able to show that each bound actually holds.

Constraints

Synthetic and public tools only. No PHI, no internal systems. The demo tools are deterministic stubs over a local corpus.

Execution had to be deterministic and auditable. Why a step ran, and why another was stopped, has to be inspectable, so the guardrails are plain control flow rather than model judgment.

The model gets no authority over what runs. The planner proposes; the allow-list, the budget, the schema checks and the human checkpoint decide.

A consequential action must not run without approval. The executor pauses at such a step and waits.

The demo has to run offline. With no key the planner falls back to a deterministic plan, so the whole flow works without a provider.

Architecture

The core is four parts. A registry holds the allow-listed tools, each with an input and an output JSON schema. A planner turns a goal into an ordered plan — a model proposes one as JSON, or a built-in deterministic planner emits the canonical plan when no provider is configured — and the plan is validated against the registry before anything runs. An executor runs the plan one step at a time: it resolves {"$ref": "stepN.field"} references against prior outputs, validates each output against its schema, retries a transient tool error up to a limit and then halts, refuses a plan beyond the step budget, and pauses at a step flagged consequential (AWAITING_CHECKPOINT) until a human approves or rejects it through a resume action. An audit layer writes every step and guardrail event to an append-only trace with per-step timing.

The guardrails are deterministic control flow over deterministic tools. There is no model call anywhere in the guardrail path, which is deliberate: the point of a guardrail is that it fires the same way every time, and a model cannot promise that.

Plan validation is where the planner’s freedom meets the registry’s rules. A plan that names a tool outside the allow-list, or wires a reference of the wrong type into a step’s input, is rejected before execution — the same static check whether the plan came from the model or from the fallback.

Key decisions

Chose plan-then-execute over a single-shot agent. The plan is emitted and inspectable before anything runs, and the executor drives it deterministically. The cost is less open-ended autonomy, which for a workflow that touches consequential actions is the point rather than the price.

Chose to let the model plan while guardrails and a human dispose. The proposed plan has no authority: the executor calls only allow-listed tools, stays inside a step budget, and stops at a consequential step. The cost is that the planner’s own quality becomes a separate concern, handled by the fallback decision below.

Chose a human checkpoint on consequential steps. A step marked consequential pauses until a person approves, so the human decides before an irreversible action rather than reviewing after it, and decisions are an append-only trail rather than an in-place status. The cost is that the run is not fully autonomous.

Chose to fall back on an invalid plan but reject a disallowed tool. When the model wires otherwise-valid tools together with a type-mismatched reference, the planner falls back to the always-valid deterministic plan and stamps the substitution in the trace. A plan naming a tool outside the allow-list is rejected instead, never quietly swapped for a safe one, because the model reaching for a tool it was not given is exactly what the allow-list exists to surface. The cost is two code paths for what looks like one failure.

Chose to persist every run as a replayable audit trace. Each step and guardrail event is written with timing, so a completed or halted run can be reconstructed. The cost is the persistence overhead, which buys the difference between a demo and something operable.

How it’s verified

make guardrails exercises all seven guardrails on a constructed plan and prints the outcome; the same behaviour is asserted case by case in the test suite. This is real output.

Scenario	Guardrail	Outcome
Normal run, checkpoint approved	—	completed, 3/3 steps
Consequential step, no approval	human checkpoint	pauses at awaiting_checkpoint
Consequential step, rejected	human checkpoint	halted
Plan over the step budget, 5 > 3	step budget	halted
Tool fails every attempt, retry ×2	retry-then-halt	failed
Tool output violates its schema	schema validation	failed
Plan calls an unregistered tool	allow-listed tools	failed

Every step is timed in the audit trace — 3 of 3 steps on a persisted run carry a recorded duration_ms, and the run carries a total_ms. The automated suite is 54 tests, including the fresh-clone case: pytest passes cold, with no seeded database.

What I’d do differently

Separate a wiring mistake from a boundary violation on day one. The rough edge showed up only with a real provider. A reviewer posing a free-form goal could hit a raw “this plan would fail at run time” error when the model wired a reference of the wrong type. The guardrail was right — the plan really would have failed — but surfacing it as a dead end was the wrong experience for a tool whose story is the guardrails rather than the planner’s wiring.

The fix was to split two cases that had been one. A type-mismatched reference is a recoverable wiring mistake, so the planner now falls back to the deterministic plan and records that it did. A disallowed tool is a signal, so it stays a hard rejection. The judgment was in not softening the allow-list to fix the interface: the two failures look alike and have to be handled in opposite directions.

Limits

It orchestrates control flow; it does not judge task quality. The eval proves the agent stops when it should, not how good the resulting brief is.

The demo tools are deterministic stubs over a local corpus, and the planner runs offline in the demo. Real tools and a real planner add their own failure modes, which is what the guardrails exist to contain.

max_steps and max_attempts are configured, not learned — per-run defaults, not tuned against a workload.

Not a multi-agent system. It is a single planner-executor loop with a human checkpoint, not agents negotiating.

Synthetic and public data and tools only. No PHI, no internal systems.

Links

The repository is linked at the top of this page. make guardrails prints the table above — every guardrail on a constructed plan, with the event it fires — and it, like the whole demo, runs offline with no key. EVAL.md lays out each guardrail and why it exists, and the run view shows a live plan pausing at its human checkpoint.