Applies toLLM applications using workflow orchestration (code, Logic Apps, Step Functions, LangGraph-style graphs) or agent frameworks (OpenAI Agents/Responses API, Anthropic tool use, Semantic Kernel, Bedrock Agents) with tools/MCP servers
Definitions
| Workflow | Agent | |
|---|---|---|
| Who decides the steps | Your code | The model, at run time |
| Predictability | High | Lower |
| Cost and latency | Bounded | Variable |
| Testing | Like normal software | Needs scenario-based evaluation |
| Fits | Known processes | Open-ended tasks where the path depends on findings |
Decision guide
| Question | Points to |
|---|---|
| Can you draw the steps as a flowchart? | Workflow |
| Does the next step depend on what was found (investigation, research, troubleshooting)? | Agent |
| Is the action irreversible or high-risk? | Workflow with human approval, or tightly bounded agent |
| Is latency/cost predictability critical? | Workflow |
| Many possible tools, varied requests? | Agent with tool selection |
Levels of autonomy
| Level | Description | Example |
|---|---|---|
| 0 | LLM step inside fixed workflow | Classify ticket → route |
| 1 | Agent suggests; human executes | Proposes remediation commands |
| 2 | Agent executes read-only tools | Queries logs, metrics, docs |
| 3 | Agent executes low-risk writes with limits | Restarts a dev service, creates a ticket |
| 4 | Agent executes high-impact actions | Changes production config (needs strong controls, approvals) |
Start at the lowest level that delivers value; move up only with evidence from evaluation and monitoring.
Controls for agents
- Step, time and token budgets (see the agent cost post).
- Read-only tools by default; write tools behind approvals.
- Tool-level kill switches (see the tool kill switch post).
- Full traces of decisions and tool calls.
- Scenario-based evaluation suites including adversarial inputs (prompt injection via tool outputs).
Hybrid is common
A workflow handles the known parts (intake, validation, approvals, notifications) and calls an agent only for the open-ended step (investigation). This keeps most of the system predictable.
Tip
Write down the autonomy level for each agent and the controls that justify it. It makes risk reviews quick and stops scope from drifting upward without a decision.
Checklist
- Steps known? Workflow considered first
- Autonomy level chosen and documented
- Budgets, approvals and kill switches in place
- Traces and scenario evaluations
- Hybrid design where only part is open-ended