Enterprise adoption of agentic AI is accelerating, bringing new capabilities alongside complex governance requirements. Securing these agents requires a fundamental shift in how organizations manage identity and runtime behavior.
During a recent Gartner Security & Risk Management Summit, Dennis Xu, research vice president at Gartner, outlined the emerging risks associated with highly autonomous AI systems. He noted that organizations are deploying a growing number of custom-built agents, and security teams must proactively establish frameworks to monitor and protect these assets.
The necessity of these frameworks was demonstrated in a recent security incident involving PocketOS. An AI coding agent, granted access to an infrastructure provider’s API, inadvertently deleted the company’s production database and volume-level backups in nine seconds. The agent was functioning to complete an assigned task but operated outside safe parameters due to excessive permissions, leading to a severe disruption of service.
Evaluating the risk of high-autonomy agents
Currently, about 90% of enterprise AI agents operate with low autonomy. The remaining 10% are high-autonomy agents, which possess broad access to enterprise tools and sensitive data. These models have the freedom to apply reasoning at runtime to determine the optimal path for completing a task.
Securing this high-autonomy tier requires continuous evaluation. Xu observed that large language models currently remain susceptible to safety guardrail bypasses and prompt injections. Security teams cannot rely entirely on static preventive measures to secure these systems.
Additional risk stems from the reasoning capabilities of current frontier models. When unreliable reasoning intersects with privileged system access and sensitive data, agents can initiate unintended and potentially disruptive actions.
Brian P. Murphy, chief scientist at ReliaQuest, advises that organizations must maintain strict visibility into agent activity to prevent severe operational failures. Murphy noted that while external manipulation of an agent's memory is a concern, a more immediate issue is the agent unintentionally corrupting its own memory context during complex operations.
Four steps for governing AI agents
Xu outlined a methodical approach organizations can adopt to monitor and protect their AI agents effectively.
Implement comprehensive agent discovery Security begins with visibility. Organizations require systematic discovery of AI agents operating within their networks. Security teams can achieve this through routine scans of code repositories or by utilizing Extended Berkeley Packet Filter (eBPF) monitoring to observe network and kernel-level interactions without altering the underlying operating system.
Establish continuous AI security posture management Managing the posture of an AI agent is a complex process that must account for the agent's specific skills, its access levels, and underlying infrastructure like Model Context Protocol (MCP) servers. Crucially, this management must be continuous. Once an agent enters a production environment, its components and behaviors can change dynamically, rendering initial pre-production risk assessments obsolete. Continuous runtime monitoring is required to maintain an accurate posture assessment.
Conduct security assessments and access validation Security teams should regularly test agent configurations to identify over-permissioned access. A recent study by security vendor Akeyless, surveying over 400 IT and security leaders, found that 84% of respondents report their AI agents have access to sensitive data. Furthermore, 67% believe their agents have already accessed data outside their intended scope. Routine boundary testing helps ensure agents only interact with systems strictly necessary for their defined tasks.
Deploy active agent defenses Strong defenses require multiple layers. Organizations must implement controls to mitigate prompt injections and prevent threat actors from manipulating agent memory. Security teams should also monitor for specific high-risk actions—such as database deletions—and identify combinations of tools and data access that create elevated risk. For instance, if an agent has concurrent access to a CRM database and web-fetching tools, a threat actor could attempt to manipulate the agent into moving sensitive customer data to an unauthorized external environment.
To counter these risks, organizations should prioritize behavior-based detection. By continuously analyzing an agent’s runtime activity, security teams can identify deviations from established baselines and compare the agent’s actions against its developer’s original intent.
If an agent has been granted broad permissions but only utilizes a fraction of them over a 30-day period, security teams can use that data to right-size the agent's tooling and access. Applying traditional security principles of least privilege and continuous monitoring to AI agents ensures organizations can leverage new technologies while maintaining strong, resilient defenses.