Now that an AI Agent can read data, make a judgment call, and even call an API to execute an action, the question isn't "can AI do this" anymore — it's "should this decision be left to AI at all."
This question is easy to overlook in the excitement of early adoption, but it's exactly the line that determines whether an AI Agent earns lasting trust or not. This article proposes a simple framework for drawing that line: where AI can be trusted to act fully on its own, where it should only propose a value for a human to confirm, and where the final decision must always stay with a person.
I. The problem: not every decision carries the same weight
Many teams designing an AI Agent start by asking the wrong question from the outset: "Should we automate this step?" — as if automation were a single yes/no choice applied uniformly across an entire business process.
In reality, a single business process usually contains decisions of very different natures:
- Decisions that only require a simple logic check (clearly right or wrong)
- Decisions that require understanding business context the data alone can't capture
- Decisions that, if wrong, can be corrected easily
- Decisions that, if wrong, cause irreversible damage (already sent to a customer, money already transferred, data already deleted, and so on)
Collapsing all of these into a single "automate or not" question is exactly what causes AI Agent projects to either over-automate and cause an incident, or become overly cautious and fail to deliver real value.
II. A framework: two axes for classifying decisions
Each individual decision within a business process can be classified along two axes.
Axis 1 — Certainty: Can this decision be judged right or wrong mechanically, based on a clear rule? Or does it involve context, judgment, or a business trade-off with no single fixed "correct answer"?
Axis 2 — Reversibility: If the decision is wrong, can it be caught and corrected easily? Or has the impact already reached outside the system — to a customer, to money, to lost data — in a way that can't be undone?
Combining these two axes gives four zones:
- Certain × Reversible → Safe to automate fully.
- Certain × Irreversible → Can still be automated, but needs extra safeguards: a tightly scoped range, audit logging, and the ability to review afterward.
- Judgment required × Reversible → AI can propose a value, but a human confirms before it's applied.
- Judgment required × Irreversible → A human should always make the final call here. AI should stick to providing information, analysis, and suggestions — never executing the action itself.
III. A worked example: four different decisions within one business process
To make this concrete, let's walk through a familiar process: handling a customer refund request on an e-commerce site. This isn't four separate systems — it's four decision points within the very same process, and each one falls into a different zone.
Step 1: Checking refund eligibility (Zone 1: Certain × Reversible)
The system checks whether the order exists, and whether the purchase date still falls within the policy's refund window (say, 30 days). This is nothing more than a date comparison — there's no room for interpretation beyond right or wrong. Even if the system gets it wrong, a staff member can simply open it and correct it, and nothing has happened to any money yet. → Safe to fully automate.
Step 2: Marking the order "eligible for refund" in the internal system (Zone 2: Certain × Irreversible)
Once an order meets every hard condition (within the window, in an eligible category, no prior refund), the system automatically switches its status to "refund approved." This is still rule-based, but it overwrites the order's status — an error here affects everything downstream. So the system tightly scopes the rule (only orders meeting *every* condition) and keeps a full audit log so it can be reviewed later. → Automated, but with a scoped range and logging.
Step 3: Proposing the specific refund amount (Zone 3: Judgment required × Reversible)
For complex cases — a customer who has already used part of a product, or an order that includes a bundled discount requiring proration — the AI proposes a refund amount along with how it was calculated. A staff member reviews the proposal and can adjust it before confirming. Since no money has moved yet, a wrong proposal costs nothing to fix on the spot. → AI proposes, a human confirms.
Step 4: Actually issuing the refund (Zone 4: Judgment required × Irreversible)
The final action is moving real money from the company's account into the customer's account or wallet. Once it's sent, recovering it is nearly impossible — especially with e-wallets or transfers to a different bank. So even if the amount was calculated correctly in Step 3, the act of "sending the money" always requires a final human check — the AI must never be the one pressing that button. → Always a human decision.
Framed this way, the question "should the refund process be automated" no longer makes sense. Three of the four steps can be automated to varying degrees. Only the final step — actually moving the money — always requires a human.
IV. What matters: the boundary belongs in code, not in a prompt
A common mistake is assuming it's enough to write in a prompt, "always check with a human before doing X." In practice, that's just a request — not an enforceable constraint.
The boundaries between the four zones above need to be implemented as real system-level constraints.
- A Tool only accepts parameters a human has already confirmed — it doesn't let the AI infer sensitive values on its own
- For Zone 4 actions, simply don't build a function the AI can call to execute them. The AI can only prepare the information needed (e.g., the refund amount); the final action — pressing execute — only exists on a screen reserved for humans, and a human must be the one to perform it
- Every action in Zones 2, 3, and 4 is logged in an audit trail showing who did it, when, and what value was chosen
In other words, this four-zone classification isn't a guideline for the AI to "try to follow" — it's the system design itself, determining which Tools exist, which don't, and how tightly scoped each one is.
V. This boundary isn't fixed — but any change should be deliberate
The boundary between "can be automated" and "needs a human decision" isn't set in stone. Once an AI Agent has built up a track record of reliability through actual operation — for example, if Zone 3 proposals have been approved without correction hundreds of times in a row — it may be worth considering moving some of those cases from Zone 3 into Zone 2.
But that has to be a deliberate design decision grounded in real operational data — not a boundary that quietly erodes over time as the AI takes on more and more work without anyone noticing.
Summary
"Should this AI Agent be automated" is always the wrong question when asked about an entire process. The right question is: "Which of the four zones does this specific decision within the process fall into — and is that boundary actually enforced by code, or just something we're 'hoping' a line of prompt text will take care of?"
Designing this boundary correctly from the start — not after an incident — is exactly what separates an AI Agent that earns lasting trust from one that gets tried once and then set aside.
