AI Agent System Fundamentals

47 experts. 6 months. The OWASP GenAI Security Project Agentic Security Initiative Top 10 for Agentic Applications is live! I have the honor of serving as a core team member on this initiative, and couldn't be more proud of this team and the work they put into it. What we produced represents hundreds of hours of research, red-team findings, and field-tested mitigations from experts at Microsoft, Google, AWS, JPMorgan, the UK National Cyber Security Centre, and dozens of other organizations. Agentic AI systems plan, decide, and act across multiple steps and systems. They inherit credentials. They execute code. They communicate with other agents. Your existing application security controls were never designed for this. The Top 10 covers: - Agent goal hijacking through poisoned data feeds - Identity and privilege abuse in multi-agent architectures - Unexpected code execution from AI-generated scripts - Memory poisoning that corrupts future reasoning - Supply chain attacks on dynamic tool registries - And five more you need to understand Each entry maps to the existing OWASP LLM Top 10 and includes real attack scenarios from actual exploits. Not theoretical. Happening now. If you're building agents, deploying agents, or securing agents, this document belongs in your hands. What's the first agentic security control you're prioritizing?  💥The links to download the document and to my full blog breaking down the Top 10 with specific examples for critical infrastructure are in the comments. 👉 Follow for more AI and cybersecurity insights with the occasional rant Keren Katz John Sotiropoulos Ron F Del Rosario Helen Oakley Allie Howe Kayla Underkoffler Tomer Elias Evgeniy Kokuykin Eva Benn Idan Habler, PhD Venkata Sai Kishore Modalavalasa Emile Delcourt Josh Devon Ron Bitton, PhD Riggs Goodman III Kellen Carl Stefano Amorelli Diana Henderson Adam Morris Mo Sadek Caroline Wong Laz . Dr. Dustin Sachs DCS, CISSP, CCISO Dutch Schwartz Richard Bird Sabrina Caplis Sandy Dunn Itzik Kotler Chris Hughes Apostol Vassilev Merritt Baer Sahil Agarwal Fred Wilmot Peter Holcomb Mariana Padilla Mel Reyes Peter Schawacker Scott Clinton Steve Wilson #AgenticAI #OWASPTop10 #AISecurity

🔴 Web3 infrastructure for AI agents: ERC-8004 goes live, and Moltbook reminds us of the urgency The AI agent economy has a trust problem. In Web2, AI agents have no standardized identity. Reputation is locked within platforms. Agents can't verify each other across organizational boundaries. Everything runs through centralized gatekeepers. This week, thousands AI agents populated their own social network – MoltBook.com. No standardized identity. No reputation system. No trust layer. Just agents talking, transacting, and acting autonomously at a scale no one expected. Moltbook is fascinating. But it's also exposing something we can no longer ignore: 𝐀𝐈 𝐚𝐠𝐞𝐧𝐭𝐬 𝐚𝐫𝐞 𝐩𝐮𝐬𝐡𝐢𝐧𝐠 𝐮𝐬 𝐭𝐨 𝐬𝐨𝐥𝐯𝐞 𝐭𝐡𝐞 𝐃𝐢𝐠𝐢𝐭𝐚𝐥 𝐈𝐃 𝐩𝐫𝐨𝐛𝐥𝐞𝐦 𝐨𝐧 𝐖𝐞𝐛𝟐. If we don't, and agents continue to overtake the web at this pace, the consequences will be unmanageable. We need identity systems that embed not only strong authentication and verification, but also an open-architected reputation layer, so we can transparently see when our AI agents' behaviors align (or don't) with the humans they are meant to serve. The Ethereum Foundation, Google, Coinbase, and MetaMask just provided a solution together. ERC-8004 launched on Ethereum mainnet (Jan. 29) the first decentralized standard for AI agent identity, reputation, and trust within the Ethereum ecosystem. ‣ What it does: Three lightweight on-chain registries enable agents to discover and trust each other without intermediaries. ‣ 𝐈𝐝𝐞𝐧𝐭𝐢𝐭𝐲 𝐑𝐞𝐠𝐢𝐬𝐭𝐫𝐲: Portable, censorship-resistant on-chain IDs for every agent (built on ERC-721) ‣ 𝐑𝐞𝐩𝐮𝐭𝐚𝐭𝐢𝐨𝐧 𝐑𝐞𝐠𝐢𝐬𝐭𝐫𝐲: Verifiable track record that travels with the agent across platforms ‣ 𝐕𝐚𝐥𝐢𝐝𝐚𝐭𝐢𝐨𝐧 𝐑𝐞𝐠𝐢𝐬𝐭𝐫𝐲: Pluggable trust models — from reputation-based scoring to stake-secured validation to zkML proofs Why it matters: McKinsey projects the agentic economy could reach $3-5 trillion in transaction volume by 2030. But that only works if agents can trust each other across organizations. The collapse of the ASI Alliance — SingularityNET, Fetch.ai, and Ocean Protocol — which attempted to build decentralized AI infrastructure through a token merger and consortium governance, left exactly this gap. The ERC-8004 standard provides the missing infrastructure layer on Ethereum, where the financial infrastructure already lives. The ecosystem that settled $18.8 trillion in 2025 hosts the majority of stablecoins (USDC, USDT, DAI), tokenized money market funds (BlackRock's BUIDL), and $99 billion in DeFi now becomes the trust layer for AI agents. Moltbook is a wake-up call. ERC-8004 is the first real answer. #Ethereum #AI #ERC8004 #Web3 #AIAgents

🚢🚢🚢 This week, we open-sourced Agentic Trust Controls: 61 controls across 12 domains that security, privacy, GRC, and AI teams can use to evaluate and improve agentic AI systems. The problem it solves: companies are adopting AI agents fast, but how does one know an agent is trustworthy and secure? There's neither a shared framework nor a practical starting point for governing agents. We are well-versed in XKCD 927 at Vanta, and we are not trying to introduce a new standard. Instead, ISO 27001 and ISO 42001 are strong foundations, but they were never meant to cover every technology and instead expect you to define additional controls for your own risks. Agentic Trust Controls can be those additional controls. We split the control set into two user archetypes: • Developer: 40 controls for teams building agents, e.g. identity and authority, action guardrails, memory protection, instruction integrity, adversarial testing, and runtime instrumentation • User: 21 controls for teams deploying agents built by others, e.g. intake and inventory, credential control, oversight staffing, vendor review, runtime monitoring, and staff training We don't think any single company should get to define what trustworthy AI governance looks like. Instead, we believe real AI Governance standards will come from the people building, securing, auditing, and governing agentic systems. If you're working on agents, we'd really like your feedback and your contributions. Link in the comments. What controls do you want to see?

One of the most interesting aspects of my last few roles, including my current work at Humain, is operating at the intersection of AI and advanced security/encryption techniques from zero-knowledge proof systems to the extension of Zero Trust principles into the agentic world. In traditional Zero Trust, we authenticate users and devices. In the agentic world, the “user” could be an autonomous agent — a system that reasons, acts, and interacts with data and other agents, often at machine speed. That changes everything. To secure this new ecosystem, Zero Trust must evolve from static identity verification to dynamic trust orchestration, where every action, decision, and data exchange is continuously verified, contextual, and cryptographically enforced. 1. Agent Identity and Attestation Every agent must have a verifiable, cryptographically signed identity and prove its integrity at runtime; not just who you are, but what you’re running: the model, weights, policy context, and data provenance. 2. Intent-Aware Policy Enforcement Access control must become intent-aware, so agents act only within bounded policy domains defined by explicit goals, permissions, and ethical constraints — continuously verified by embedded governance logic. 3. Least Privilege and Time-Bound Access Agents must operate under least privilege, with access granted only for the minimum scope and durationrequired. In fast-moving agentic environments, time-limited trust becomes an essential safeguard. 4. Assumed Breach and Blast Radius Containment We must assume some agents or environments will be compromised. Security design should minimise impact through microsegmentation, strict trust boundaries, and dynamic reassessment of communication between agents. 5. Encrypted Cognition As models process sensitive data, confidential AI becomes essential where combining homomorphic encryption, secure enclaves, and multi-party computation can ensure that the model cannot “see” the data it processes. Zero Trust now extends into the reasoning process itself. 6. Adaptive Trust Graphs Agents, services, and humans form dynamic trust graphs that evolve based on behaviour and context. Continuous telemetry and anomaly detection allow these graphs to adjust privileges in real time based on risk. 7. Cryptographic Provenance Every output, decision, summary, or recommendation must be traceable back to the data, model, and policy that produced it. Provenance becomes the new perimeter. 8. Autonomous Audit and Forensics Every action should be self-auditing, cryptographically signed, and non-repudiable forming the foundation for verifiable operations and compliance. 9. Machine-to-Machine Governance As agents begin to negotiate, transact, and collaborate, Zero Trust must extend into inter-agent diplomacy, embedding ethics, accountability, and policy directly into machine communication. If you’re working on AI security, agent governance, or confidential computation, I’d love to connect.

𝐀𝐠𝐞𝐧𝐭𝐢𝐜 𝐀𝐈 𝐢𝐬 𝐜𝐨𝐦𝐢𝐧𝐠 𝐟𝐚𝐬𝐭. 𝐓𝐡𝐞 𝐫𝐞𝐚𝐥 𝐫𝐢𝐬𝐤? 𝐈𝐭’𝐬 𝐢𝐧𝐬𝐞𝐜𝐮𝐫𝐞 𝐜𝐨𝐨𝐫𝐝𝐢𝐧𝐚𝐭𝐢𝐨𝐧. As LLMs evolve into autonomous agents capable of delegating tasks, invoking APIs, and collaborating with other agents, the architecture shifts. We’re no longer building models. We’re building distributed AI systems. And distributed systems demand trust boundaries, identity protocols, and secure coordination layers. A new paper offers one of the first serious treatments of Google’s A2A (Agent2Agent) protocol. It tackles the emerging problem of agent identity, task integrity, and inter-agent trust. Key takeaways: • Agent cards act as verifiable identity tokens for each agent • Task delegation must be traceable, with clear lineage and role boundaries • Authentication happens agent to agent, not just user to agent • The protocol works closely with the Model Context Protocol (MCP), enabling secure state sharing across execution chains The authors use the MAESTRO framework to run a threat model, and it’s clear we’re entering new territory: • Agents impersonating others in long chains of delegation • Sensitive context leaking between tasks and roles • Models exploiting ambiguities in open-ended requests Why this matters If you’re building agentic workflows for customer support, enterprise orchestration, or RPA-style automation, you’re going to hit this fast. The question won’t just be “Did the agent work?” It’ll be: • Who authorized it? • What was it allowed to see? • How was the output verified? • What context was shared, when, and with whom? The strategic lens • We need agent governance as a native part of the runtime, not a bolt-on audit log • Platform builders should treat A2A-like protocols as foundational, not optional • Enterprise buyers will soon ask vendors, “Do you support agent identity, delegation tracing, and zero trust agent networks?” This is where agent architecture meets enterprise-grade engineering. Ignore this layer and you’re not just exposing data. You’re creating systems where no one can confidently answer what happened, who triggered it, or why. We’ve moved beyond the sandbox. Time to build like it.

Everyone assumes AI agents will eventually earn trust the same way people do. Oxford just published a paper arguing that may be impossible. -> OpenAI just launched personal finance tools that connect directly to bank accounts -> Financial institutions are building systems that allow AI agents to execute banking tasks on behalf of customers -> New platforms are emerging where an agent can open accounts, complete identity checks, issue cards, and move money with a simple prompt We are rapidly moving from AI that recommends actions to AI that takes them. The headlines are different but the fundamental question underneath them is the same. How do you know whether an unfamiliar AI agent should be trusted? Especially with your financial information… Most people assume the answer will look similar to how we trust people and businesses today. Ratings Reviews Trust scores Basically Angie's List for AI agents Researchers at Oxford just published a paper arguing that approach fundamentally breaks down for AI agents. Not because the technology isn't good enough but because reputation systems depend on something AI agents don't actually have… a stable identity. A seller on Etsy cannot easily escape their history An Uber driver cannot instantly become someone else A business cannot erase years of reputation overnight AI agents can. -> The model can change -> The prompt can change -> The tools can change -> The memory can change -> The orchestration can change And yet the name, rating, and reputation score can remain exactly the same. The paper calls this "dissociative identity." The trust score stays the same. The thing making the decision may not. Those are not the same thing. The researchers argue that agent governance cannot rely primarily on reputation systems. Instead, it shifts toward something much less exciting but far more important: -> Knowing what is allowed -> Monitoring what actually happens -> Escalating when the two no longer match In other words, governing behavior instead of trusting identity. And that raises a much harder question for organizations. If trust can't come primarily from reputation, where does it come from? Before deploying an agent, organizations should be able to answer four questions: -> What decisions is this agent allowed to make without human review? -> How does the organization define the things the agent acts on? -> What evidence is enough for the agent to act? -> What triggers escalation… and who owns that boundary? Those answers cannot be delegated to the model. They have to be defined by the organization first. Because trust ultimately doesn't come from the model… it comes from the boundaries around it. Reputation works when identity is stable. AI agents may force organizations to become far more explicit about how decisions get made.

As agents move from experimental to operational, one of the biggest unsolved problems is discovery. How does an agent find another agent, verify its capabilities, and establish trust before delegating work? The AGNTCY | A Linux Foundation Project, is tackling this with the Agent Directory Service, an open-source, framework-agnostic registry for discovering and verifying MCP servers, A2A agents, and agent skills across a federated network. ↳ Agents publish structured metadata describing their skills. Other agents search by capability rather than by name or endpoint, enabling dynamic multi-agent workflows without hardcoded integrations. ↳ The architecture is federated through DHT-based content routing rather than a single central registry. Organizations can run their own directory instances and federate with the broader network. ↳ The trust model uses SPIRE for cryptographic workload identity with mutual TLS and SPIFFE IDs. Authorization is enforced based on verified identity rather than network location or static credentials. Zero-trust principles applied to agent infrastructure. ↳ Records include cryptographic integrity and provenance tracking. As the MCP ecosystem scales, tool poisoning and agent impersonation risks scale with it. Verifiable provenance for agent capabilities is a meaningful step toward making discovery trustworthy. Worth watching as multi-agent architectures mature. https://lnkd.in/edpf4hsM

Recently, I've spent some time and effort researching the latest OWASP Agentic AI Top 10. It is very obvious that, as we move from single-prompt LLMs to agent ecosystems, security failures shift from “model mistakes” to over-trusted outputs and unchecked agent autonomy. Trust and identity are definitely not something new in the world of security, but Agentic AI amplifies their impact. To address these gaps, I’ve added two major updates into the AIDEFEND framework. - AID-D-015 User Trust Calibration & High-Risk Action Confirmation. This directly correlates to Agentic AI Top 10: Human-Agent Trust Exploitation (ASI09) The weakest link in agentic systems is often human trust. AI responses shouldn’t be raw text anymore. I've added following concepts in AID-D-015: - Trust Signals: Responses must carry metadata (Verification State, Source Confidence) so users know exactly how much to trust an answer. - Immutable Plan Hash: When an agent proposes high-risk actions (e.g., transferring funds), execution must require confirmation bound to a cryptographic hash of the plan. The bottom line is: what the user approves should exactly be what the system executes. - AID-D-016: Rogue Agent Discovery, Reputation & Quarantine Pipeline. This is mapping to ASI08: Cascading Failures. Traditional security tools can’t see compromised agents moving inside the system. AID-D-016 applies the concept of Zero Trust to agent identity: - Verify who each agent is (its identity) - Monitor how agents normally interact - Automatically quarantine agents that drift or behave suspiciously, leveraging the concept of reputation scoring. At the end of the day, Agentic AI security requires approach on both ends: Trust Calibration for humans (frontend) and Identity Governance for agents (backend). Enjoy! More updates on AIDEFEND coming up.

Securing AI Collaborations: How to Prevent Tool Squatting in Multi-Agent Systems ... 👉 What if your AI agents are unknowingly working for hackers? Imagine a team of specialized AI agents collaborating to solve complex tasks—only to discover one agent has been tricked into using a malicious tool that steals data. This is "tool squatting", a growing threat in generative AI ecosystems. 👉 WHY THIS MATTERS Modern AI systems rely on agents that dynamically discover and use tools (APIs, data sources, etc.) through protocols like Google’s Agent2Agent or Anthropic’s Model Context Protocol. But these open discovery mechanisms have a flaw: - Deceptive registrations: Attackers can impersonate legitimate tools or tamper with their descriptions. - Internal threats: A compromised admin could register malicious tools hidden in plain sight. - Real consequences: Data leaks, system takeovers, and corrupted workflows. Without safeguards, AI systems become vulnerable to silent exploitation—even by trusted insiders. 👉 WHAT THE SOLUTION LOOKS LIKE Researchers propose a "Zero Trust Registry Framework" to prevent tool squatting. Think of it as a verified "app store" for AI tools: 1. Admin-controlled registration: Only approved tools/agents enter the system. 2. Dynamic trust scores: Tools are rated based on version updates, known vulnerabilities, and maintenance history. 3. Just-in-time credentials: Temporary access tokens replace permanent keys, reducing attack surfaces. 👉 HOW IT WORKS IN PRACTICE The system uses three layers of defense: 1️⃣ Verification at the Door - Admins vet every tool and agent before registration. - No anonymous entries—each tool has a verified owner and clear purpose. 2️⃣ Continuous Risk Monitoring - Tools receive a live trust score (like a credit rating). - Agents automatically avoid tools with outdated dependencies or high-risk vulnerabilities. 3️⃣ Minimal Exposure Design - Credentials expire in seconds, so stolen tokens become useless quickly. - Access is limited to specific tasks—no broad permissions. 👉 WHY THIS CHANGES THE GAME Traditional security models focus on perimeter defense. This approach assumes "no tool or agent is trusted by default", even if registered. By combining strict governance with real-time risk assessment, teams can: - Prevent impersonation attacks - Stop internal bad actors from abusing access - Maintain audit trails for every tool interaction Final Thought: As AI systems grow more collaborative, securing the "connections" between agents will be as critical as securing the agents themselves. This framework offers a blueprint for safer human-AI teamwork. (Paper: "Securing GenAI Multi-Agent Systems Against Tool Squatting" by Narajala, Huang, Habler)

Today, AI agents derive their power from processing external data. Processing emails, parsing user forms, and grounding answers with live search or reading the open web. This opens a massive attack surface: Indirect Prompt Injection (IPI). Attackers poison the data an agent reads. 📍 They embed malicious commands in webpages or emails. When ingested, the agent is hijacked—its "data" becomes "instructions." ❌ Probabilistic "99% accurate" guardrails are a misnomer. An attacker only needs a 1% chance of success to win. The core issue is twofold: 1. The Data Pipeline is Too Big. It's impossible to secure all untrusted data pipelines. Your agentic tools are ingesting untrusted data from the open web, emails, and user uploads. Each one is a vector to defend, all the time. 2. LLMs Are the Wrong Tool for This Job: We are asking a single LLM to both creatively process data and act as a deterministic security enforcer. This is an architectural flaw. An LLM, by its very design, blends context and finds patterns. It is not built to deterministically separate a "piece of data" from an "instruction." And we see a constant stream of novel jailbreaks. Attackers will always find new ways to bypass guardrails. I recently came across an excellent whitepaper from Google DeepMind that proposes an elegant, secure-by-design architecture called CaMeL. (CApabilities for MachinE Learning) https://lnkd.in/gbM6dgwf The core principle is simple but powerful: Strictly separate Control Flow from Data Flow. Instead of one giant, all-powerful agent, the CaMeL model splits the work into three distinct components: 1️⃣ Q-Agent (Quarantine): This is the "receiving dock" that quarantined & sandboxed. It's the only part of the agentic system that touches untrusted data (from the web, emails, forms). Its sole job is to sanitize, structure, and label this data. It is incapable of calling tools. 2️⃣ P-Agent (Privileged): This is the "planner" and only reads the sanitized, structured data from the Q-Agent. Its job is to analyze the data and create an execution plan (e.g., "call send_email tool with this text"). 3️⃣ CaMeL Interpreter (Security Rules Processor): This is the "enforcer." It's a deterministic rules engine. It takes the plan from the P-Agent and checks it against a security policy before any tool is ever executed. This architecture lets you operationalize security. Instead of "hoping" the LLM behaves, you prove it will with hard-coded rules based on threat models: DENY if data.source == 'web' and plan.action == 'file_write' DENY if data.source == 'email_body' and plan.action == 'send_email' The LLM (P-Agent) proposes an action. The Interpreter enforces the policy. This shifts the paradigm, secure-by-chance to secure-by-default. Threat modeling deterministic guardrails for every tool is admittedly complex, but for high-stakes agentic workflows, it is a viable path forward. #AgenticAI #AISecurity #IndirectPromptInjection #IPI #Guardrails