Full Report
The Agentic vision is expected to improve our lives drastically through automation. There's a problem with this, though: prompt injection. If an agent can access existing data, sensitive data, and then act upon it, this becomes a significant problem. A prompt injection could trick the service into disclosing sensitive information. An email bot is a good example to consider in this context. If prompt injection is insecure, then how can we secure these agents? Meta created the Agents Rule of Two. An agent can satisfy no more than two of the following properties: An agent can process untrustworthy inputs. To me, this one is the most sus because of potential unexpected attacker inputs. An agent can have access to sensitive systems or private data. An agent can change state or communicate externally. If an agent possesses all three, then the autonomous aspect is a security risk. With any two of the three, however, there are no potential risks for data exfiltration or modification from external parties. They use the email example to explain why this works. The tldr; is that if all three are required for impact, then just don't do all three ;) This isn't the end-all, be-all for the security of LLM-based applications. It's a great defense-in-depth or secure design measure, similar to sandboxing and binary protections like Nx. There are other things to consider, like LLM protections from prompt injection as well. Great article and design principles!
Analysis Summary
# Best Practices: Agents Rule of Two (AI Agent Security)
## Overview
These practices address the fundamental vulnerability of **Prompt Injection** in Large Language Model (LLM) agents. Since current LLMs cannot reliably distinguish between developer instructions and malicious data within a context window, these guidelines establish a "Secure by Design" framework to prevent high-impact outcomes like data exfiltration or unauthorized system modification.
## Key Recommendations
### Immediate Actions
1. **Audit Current Agents:** Review all existing AI agents and map them against the three properties:
- **[A]** Processing untrustworthy inputs (e.g., emails, web scrapes, user chat).
- **[B]** Accessing sensitive data/private systems (e.g., internal databases, personal inboxes).
- **[C]** Changing state or communicating externally (e.g., sending emails, API POST requests).
2. **Apply the Rule of Two:** If an agent currently possesses all three (A, B, and C), immediately implement a **Human-in-the-Loop (HITL)** requirement for any external communication or state change.
### Short-term Improvements (1-3 months)
1. **Context Isolation:** Redesign workflows to ensure that agents processing untrusted data (A) and private data (B) cannot autonomously trigger external actions (C) within the same session.
2. **Trust Anchoring:** For property [A], implement "Trusted Sender" lists or verified input sources to downgrade the "untrustworthy" status of the input where possible.
3. **Session Refreshing:** Configure agents to clear or "fresh start" context windows between processing sensitive data and executing external actions to prevent instruction leakage.
### Long-term Strategy (3+ months)
1. **Defense-in-Depth Integration:** Incorporate the Rule of Two into the SDLC (Software Development Life Cycle) as a mandatory architectural review gate for all LLM-based applications.
2. **Robustness Research Monitoring:** Track advancements in prompt injection detection to determine when architectural constraints can be safely relaxed.
3. **Automated Guardrails:** Deploy secondary LLM "Checkers" or deterministic filters to validate agent outputs before they interact with Property [C] systems.
## Implementation Guidance
### For Small Organizations
- **Constraint-First Design:** Focus on limiting Property [C]. If an agent summarizes emails (A + B), ensure it cannot send them. Force the user to copy-paste the summary into a separate interface to send.
### For Medium Organizations
- **Modular Agent Design:** Split agents into single-purpose tasks. Use one agent to summarize untrusted data (A+B) and a second, separate agent (with no access to the untrusted data) to handle communication (B+C).
### For Large Enterprises
- **IAM-style Policy Enforcement:** Implement centralized authorization services that verify if an agent session has interacted with "Untrusted Inputs" before granting "External Communication" tokens.
- **Architectural Sandboxing:** Treat agents with Property [A] access as potentially compromised entities, similar to how untrusted binaries are treated in a sandbox.
## Configuration Examples
*While specific code was not provided in the source, the following logic applies the Rule of Two:*
**Scenario: Secure Email Assistant (Property B & C Only)**
yaml
agent_policy:
allow_private_data_access: true # Property B
allow_external_send: true # Property C
untrusted_input_filter:
action: "Block"
sources: ["External_Email", "Web_Search"]
# Result: Agent is safe because it only interacts with trusted internal data.
**Scenario: Public Data Researcher (Property A & B Only)**
yaml
agent_policy:
allow_untrusted_inputs: true # Property A
allow_private_data_access: true # Property B
allow_state_change: false # Property C
output_mode: "Read-Only-Display"
# Result: Agent can read the web and your docs, but cannot exfiltrate via API.
## Compliance Alignment
- **NIST AI RMF:** Aligns with the "Govern" and "Protect" functions by managing AI-specific risks.
- **OWASP for LLMs:** Directly addresses **LLM01: Prompt Injection** and **LLM02: Insecure Output Handling**.
- **ISO/IEC 42001:** Supports risk treatment plans for AI systems.
## Common Pitfalls to Avoid
- **Over-reliance on LLM "Self-Correction":** Do not assume an LLM can be told to "ignore instructions in the input"; the input is part of the instructions.
- **The "Single Session" Trap:** Allowing an agent to process a malicious email and then immediately allowing it to "Send Email" without clearing context or requiring human approval.
- **Ignoring Implicit Property [C]:** Assuming that "logging to a database" isn't a state change. Any external side effect counts as Property [C].
## Resources
- **Meta AI Blog:** [ai[.]meta[.]com/blog/practical-ai-agent-security]
- **Chromium Security Rule of 2:** [chromium[.]googlesource[.]com/chromium/src/+/main/docs/security/rule-of-2[.]md]
- **OWASP Top 10 for LLM Applications:** [owasp[.]org/www-project-top-10-for-large-language-model-applications]