The architecture question hiding behind every AI deployment
When an enterprise deploys an AI agent, there's a fundamental design choice that determines everything downstream: who makes the decisions — the LLM or the rules?
This isn't a theoretical question. It determines whether the system can be audited, whether failures can be traced, whether decisions can be rolled back, and whether the whole thing can be explained to a regulator. It's the difference between a system that produces predictable outcomes and a system that produces plausible outputs.
Architecture A: The LLM decides
In the most common agent architecture today, the LLM is the decision-maker. It receives a task, reasons about it, selects tools, determines the order of operations, and executes. Frameworks like LangChain, CrewAI, and AutoGen follow this pattern. The LLM is the brain, the planner, and the executor.
This works remarkably well in demos. It also works well for low-stakes, exploratory tasks where occasional errors are acceptable. The problem emerges when you need the same system to behave consistently, explain itself, and operate within defined boundaries.
The CIO.com reporting on agentic AI trust issues captures this: "Most enterprises today can stand up an agent but very few can explain, constrain, and coordinate a swarm of them." A product manager at Writer (an agent-building platform) observed that many organizations view agents as similar to API calls with predictable outputs, when in reality they behave more like "junior interns."
IBM's Maryam Ashoori described the failure mode precisely: "If the model hallucinates and takes the wrong tool, and that tool has access to unauthorized data, then you have a data leak." The issue isn't hallucination per se — it's that a hallucination at the decision layer has operational consequences that are difficult to detect, trace, and reverse.
Google Cloud's 2025 review noted that "deploying agents has become less a software problem and more a governance challenge. In complex workflows with multiple agents, it's difficult to isolate which agent drove success or caused failure."
The fundamental problem: when the LLM decides, the decision boundary is the model itself — opaque, non-deterministic, and version-dependent. You can observe what it did. You cannot guarantee what it will do.
Architecture B: The LLM extracts, rules decide
The alternative architecture separates two distinct functions: understanding and deciding.
The LLM handles understanding — interpreting natural language, extracting structured data from unstructured input, classifying intent, identifying entities. This is what LLMs are demonstrably good at. The output is structured data that conforms to a typed contract.
The decision engine handles deciding — applying explicit rules to that structured data to determine what action to take. Which rules fire. What the thresholds are. Where to route. Whether to approve or escalate. These rules exist as versioned artifacts — decision tables, policy definitions — that can be read, tested, and audited independently of any AI model.
The practical difference:
In Architecture A, when you ask "why did the agent route this request to the fraud team?", the honest answer is: "The LLM determined this was the best action based on its training and the context provided." The explanation is a reconstruction — a plausible story about what the model probably considered.
In Architecture B, the answer is: "The LLM extracted these fields from the input: amount: $47,000, country: Nigeria, account_age: 3 days. Rule 12 in the fraud_screening table (v2.4) specifies that amount > $10,000 AND account_age < 7 days triggers fraud review. The rule was last modified by analyst name on date." The explanation is a fact — traceable to a specific rule, a specific version, a specific person.
Why this distinction matters more in 2026
Three converging forces make this architectural choice urgent:
1. Regulatory requirements are being phased in. The EU AI Act (Regulation 2024/1689) becomes broadly applicable from 2 August 2026, with earlier dates for specific chapters. Article 86 establishes a right — for individuals affected by decisions based on Annex III high-risk AI systems with significant effects on health, safety, or fundamental rights — to obtain from the deployer clear and meaningful explanations of the AI system's role and the main elements of the decision.
The distinction between intrinsic and post-hoc explainability matters here. A TechPolicy.Press analysis of the GDPR and AI Act argues that intrinsic methods (where decision logic can be traced directly) and post-hoc methods (approximations applied to opaque models) differ materially for contestability — and warns that post-hoc explanations can be unreliable, falling short of the protections possible when human decisions are contested.
When the model is the decision boundary, explanations are typically post-hoc — reconstructions of what the model probably considered. When decisions flow through explicit rules, explanations are intrinsic — traceable to a specific rule and version. As enforcement of the AI Act begins (first obligations already apply, with the bulk taking effect August 2026), this architectural distinction has direct compliance implications.
2. Agent sprawl is creating governance chaos. IBM reports that by late 2025, enterprises found themselves with dozens or hundreds of agents on different platforms, built by different teams. Deloitte found only 1 in 5 companies has mature governance for autonomous agents. When decisions are made by LLMs inside black-box agents, governing a fleet of them is practically impossible. When decisions are made by explicit rules that exist as versioned artifacts, governance becomes a version control problem — still hard, but tractable.
3. Failure recovery requires determinism. When an AI agent makes an incorrect decision, recovery depends on understanding exactly what happened and undoing the consequences. If the LLM chose to call a tool that triggered an irreversible action, and you can't explain why it chose that tool, you can't build systematic failure recovery. If rules made the decision, the failure trace is exact: this input, this rule, this version, this outcome. And the fix is equally exact: change the rule, test, deploy.
This isn't "anti-AI"
Separating "understand" from "decide" doesn't reduce the role of AI. It focuses the AI on what it does best — handling ambiguity, interpreting natural language, extracting structure from chaos — while keeping the decision boundary explicit and auditable.
The LLM remains essential. Without it, you're back to rigid forms and structured input — which is where many enterprise systems are stuck. The LLM is what makes the system able to handle real-world, messy, human input.
But the LLM's output goes through a checkpoint — typed, structured, validated — before it affects any decision. This is the same principle as input validation in security: you don't trust raw user input, and you shouldn't trust raw model output in a system where decisions have consequences.
Cleanlab's survey of production AI teams found that more than half plan to focus on reducing hallucinations, and 42% of regulated enterprises plan to introduce manager features such as approvals and review controls. These are symptoms of the same problem: when the LLM is the decision-maker, you need extensive human oversight to compensate for its unpredictability. When rules make decisions, the oversight shifts from "did the AI do something wrong?" to "are the rules correct?" — a more tractable, more scalable question.
The choice
Every AI agent in production sits somewhere on a spectrum between "the LLM decides everything" and "the LLM extracts, rules decide."
Many agent frameworks default to the left side of that spectrum — because it's faster to build, more impressive in demos, and requires less upfront design. The enterprises that report success in production, particularly in regulated environments, tend to move toward the right — because it's auditable, testable, and governable.
This isn't a technology preference. It's a risk management decision. And like all risk management decisions, it should be made deliberately, not by default.