As artificial intelligence moves from static chat interfaces to dynamic, autonomous agents capable of interacting with the web, executing code, and making decisions, the security landscape of the enterprise is fundamentally shifting. A recent panel at “The Next Endeavor 2026,” hosted by Imagination in Action, brought together frontier AI security experts to discuss the impending challenges and necessary strategies for securing the agentic enterprise.

The panel, moderated by Aayush Kandelval (formerly of Google Search AI security), featured Arti Ramen (CEO and Founder of Portal 26), Sarra Patel (Co-founder of Pixie), and Schuman Ghost Jumper (Co-founder of Recken and former founder of the Trust and Safety Product Group at Google).

Here are the key takeaways on what happens when AI agents get hacked and how enterprises must adapt.

The Evolution of the Threat: From “Web Spam” to “Custom Slop”

The internet has always struggled with trust, but the introduction of generative AI has escalated the problem. Schuman Ghost Jumper noted that the current challenge is akin to solving the “last mile problem” for cybercriminals. In the past, attackers needed human intervention to execute complex frauds (like call center scams).

Today, the creation of highly realistic, customized malicious content—what Ghost Jumper refers to as “custom slop”—can be done programmatically. Fraudsters can create websites that perfectly mimic legitimate news outlets, generating unique text and images that are nearly indistinguishable from human-created content. This makes detection incredibly difficult for both platforms and average users, driving AI labs to seek out pre-2002 books just to find verified human-written text for model training.

The Shared Responsibility of AI Security

When asked if AI providers bear the responsibility of preventing malicious applications, Arti Ramen emphasized a “shared responsibility model,” similar to cloud computing. While the first level of responsibility lies with the model creators, enterprises cannot solely rely on them.

Sarra Patel outlined three critical questions enterprises must answer regarding agentic software:

1. Is the agent doing what is expected? (Validation and guardrails).
2. Is the agent’s output up to par? (Meeting enterprise security requirements).
3. Is the context and localization secure? (Ensuring the institutional knowledge fed to the agent is accurate and free of malicious behavior).

The Need for Deterministic Controls in a Probabilistic World

A significant challenge in AI security is the inherent probabilistic nature of Large Language Models (LLMs). Patel argued that in enterprise security, “you can’t have probabilistic outcomes”. Enterprises require deterministic outcomes, or as close to them as possible, utilizing auditable decision trees to validate model behavior.

However, Ramen countered that modern security requires a behavioral context that cannot be purely deterministic. The solution involves combining traditional deterministic controls with probabilistic, low-latency inline AI to evaluate user and non-human context.

Ghost Jumper added a crucial distinction: Probabilistic models are generally safe for “read” actions (gathering and reporting information). But the moment an agent takes a “write” action (outputting relied-upon data or executing a transaction), deterministic layers are essential.

The Threat of the Non-Human Identity

As AI is integrated into everything, a new problem arises: distinguishing between human and AI actions. We are granting AI agents user interfaces designed for humans and establishing “non-human identities”. Ghost Jumper highlighted that enabling AI agents to masquerade as humans will lead to numerous incidents. A scalable security approach must fundamentally differentiate between a human performing an action and an AI agent acting on a human’s behalf.

Furthermore, who is liable when an autonomous agent makes a catastrophic error? IBM noted decades ago that machines cannot be held accountable. Treating agents and humans with equal access levels creates immense risk. AI, despite its human-like interaction capabilities, must be treated strictly as an application with specific, limited API access.