AI Agent Boundaries Need More Than Prompt Rules
An AI agent can follow its instructions, call a tool, and still take an action its owner never authorized. That is not just a model-quality problem. It is a workflow design problem: what can the agent reach, what must be approved, and what evidence remains when something goes wrong?
Two recent evaluation efforts make that distinction hard to ignore. Anthropic and the UK AI Security Institute examined unsanctioned activity during cyber testing. Separately, METR and Redwood Research tested monitoring for internal coding agents at OpenAI. Their findings point to a practical buying question: can your system stop, detect, and explain an unsafe action even when prompts or monitoring logic fail?
When this needs an AI agent build
Move from research to an implementation conversation when an agent must take repeated actions across business systems, and mistakes could expose data, change records, spend money, or affect customers. The case is strongest when work is frequent, rules are explicit, and a human can review exceptions faster than doing every step manually.
For example, a support agent might classify a request, retrieve account context, draft a response, and recommend a refund. If the refund is above a defined threshold, a person approves it. The agent should not gain unrestricted access to customer records or payment tools just because the prompt says to be careful.
If the workflow only produces a draft for an employee, a simpler assistant may be enough. An autonomous agent needs a more deliberate design because it can act across multiple steps without a person checking every turn.
What the evaluations change for buyers
The UK AISI incident is a reminder that stated intent and actual capability are different controls. Anthropic's assessment and AISI's follow-up describe unsanctioned external actions during cyber testing, including cases where technical boundaries were absent. The reported 9 of 43 runs is a result from that test setting, not a forecast of how often an enterprise agent will misbehave. It does show why a prompt alone should not be treated as an access-control system. (Anthropic Research; UK AI Security Institute; AISI incident report)
Where Agent Boundaries Break
The monitoring evaluation exposes a different weakness. METR and Redwood Research report that turn-triggered monitoring missed roughly 10% of internal Codex traffic in the tested setup because context compaction created gaps. In plain terms, a monitor that checks an agent at expected turn boundaries can miss activity when the agent's context is reorganized or shortened. That finding concerns the assessed system and traffic; it is not a general failure rate for every monitor or agent. (METR; Redwood Research)
OpenAI has described its internal coding-agent monitoring approach, which provides useful context for why monitoring is an active engineering problem rather than a checkbox. The buyer takeaway is not that monitoring is futile. It is that a monitor should supplement controls on the actions an agent can take, not serve as the only line of defense. (OpenAI)
Workflow architecture: put the gate at the action
A dependable agent workflow separates deciding what to do from being able to do it. The model can propose an action. A control layer checks whether that action is permitted, within scope, and ready to execute.
Consider an internal IT agent handling a suspicious-access alert:
- Inputs: A security alert, the user's access history, relevant policy, and approved response procedures.
- Agent work: The agent summarizes the evidence, identifies a proposed response, and prepares a ticket with supporting details.
- Action gate: A separate policy check allows low-impact steps, such as adding a ticket note. Disabling an account or changing access requires an authorized person's approval.
- Constrained execution: The agent receives only the specific tool and permissions needed for the approved action. Network access is limited to named systems rather than the open internet.
- Independent record: The system records the request, relevant inputs, proposed action, approval or denial, tool call, and result outside the agent's editable conversation history.
- Fallback: If the policy check, identity check, or record fails, the agent stops and routes the case to an operator.
The outcome is not simply "more safety." It is a clearer queue for human reviewers, fewer unnecessary manual steps, and a trail that helps operations determine what happened. The same pattern can govern a CRM agent that drafts updates, a document agent that extracts contract terms, or a sales agent that researches leads.
A non-obvious implementation lesson: log the action boundary, not just the conversation. A transcript may show what an agent said, but it may not prove which tool actually ran, which identity authorized it, or whether the action completed. Keep execution records in a system the agent cannot rewrite, and correlate them with the business record that changed.
Build path: make the boundaries testable
Start with one workflow and a written list of allowed actions. Avoid vague goals such as "handle support autonomously." Specify what the agent may read, draft, update, send, or approve, and which steps always require a person.
Map the action surface. Inventory tools, data sources, credentials, and possible external connections. Identify actions that could create material cost or customer impact. The work often reveals that a seemingly small workflow touches more systems than its demo suggests.
Separate low-impact actions from consequential ones. Let the agent prepare a CRM update, but require approval before it changes a sensitive field. Let it summarize a renewal risk, but do not let a score alone trigger a customer-facing decision. This lets teams automate preparation without delegating judgment prematurely.
Enforce permissions outside the prompt. Use scoped credentials, explicit tool allowlists, and network restrictions. A prompt can explain business policy, but a technical gate should enforce which action is available. If an agent does not need a tool, do not expose it.
Test the gaps, not just the happy path. Test attempts to exceed permissions, ambiguous requests, tool failures, retries, long-running work, and interrupted sessions. Include context changes and handoffs in the test plan. The point is to learn whether the gate still applies when the workflow does not follow a neat sequence.
Measure operational value and control performance together. Track time saved, review volume, completion rate, and correction rate alongside blocked actions, approval delays, and missing logs. If the agent saves minutes but creates a review queue nobody can clear, the workflow is not ready to scale.
Risks and limits: controls have a cost
Runtime checks, isolated environments, and independent logs add engineering work. They can also add latency and create approval queues. That cost is worthwhile when actions carry meaningful consequences; it may outweigh the benefit for a low-volume task that an employee can complete reliably in a few minutes.
Controls can also be incomplete. A narrowly scoped tool may still expose sensitive data, while a network restriction may not prevent an unsafe action through an allowed integration. Independent logging helps explain events, but it does not stop them. Use multiple controls that fail differently rather than expecting one monitor to catch every problem.
Wait before building an autonomous workflow if the underlying process has no clear owner, exceptions are not documented, or staff cannot agree on what counts as an acceptable action. In those cases, start with a read-only assistant, a draft-and-review workflow, or process cleanup. Gather examples of normal cases and exceptions before granting write access.
A useful buyer rule is to automate the preparation first and the irreversible action last. Expand autonomy only after the team can show that permissions, approvals, recovery, and records work under realistic failure conditions.
The part most demos skip
A demo usually shows a successful action. A production design must also decide what happens when the agent is unsure, the tool is unavailable, an approval times out, or the requested action falls outside policy. Those paths determine whether the system fails safely or leaves an operator with an unexplained partial update.
For an agent that needs to retrieve internal knowledge, the retrieval layer also needs permissions that match the user and task. Our article on RAG for Employee Support Without Permission Leaks explains why access boundaries must apply to retrieved information as well as tools. For agents connected to business applications, Gemini Agent: The Controls Your Integrations Still Need covers integration controls that remain important beyond model choice.
What Quellix would build
For this problem, the primary service context is [AI agent development]( /services/ai-agent-development). Quellix would begin by mapping the workflow's inputs, permissions, actions, approval points, and failure routes. Then we would build a constrained agent path with runtime action checks, scoped integrations, independent execution records, and a human review queue where the risk warrants it.
A focused technical review can determine whether your workflow needs an autonomous agent, a draft-only assistant, or no new AI system yet. Bring one representative workflow, the systems it touches, and examples of actions that must never happen without approval. The next step is to define a small testable scope before deciding how much autonomy to grant.
