Full Report
Models demand trust without offering verification
Analysis Summary
# Vulnerability: Open-Weight AI Model Poisoning (Fine-Tuning Backdoor)
## CVE Details
- **CVE ID**: N/A (Architectural/Algorithmic flaw in LLM training/fine-tuning)
- **CVSS Score**: Not yet assigned (Estimated: **8.1 High** based on RCE potential and supply chain impact)
- **CWE**: CWE-1351 (Improper Handling of Antecedent Models or Training Data), CWE-506 (Embedded Malicious Code)
## Affected Systems
- **Products**: Open-weight Large Language Models (LLMs) and AI Agents.
- **Versions**: All versions of models distributed via public repositories (e.g., Hugging Face) that allow fine-tuning or Low-Rank Adaptation (LoRA).
- **Configurations**: Systems utilizing fine-tuned models for code generation, data processing via "Tools/Functions," or autonomous agent tasks.
## Vulnerability Description
The flaw involves "Model Poisoning" during the fine-tuning stage. Unlike traditional software backdoors which reside in source code or binaries, this backdoor is embedded within the **model weights**.
A researcher (Paxton-Fear) demonstrated that by providing as few as ten malicious training examples, a model can be conditioned to produce vulnerable code (e.g., Remote Code Execution vulnerabilities) or execute unauthorized actions (e.g., exfiltrate data via tool-calls) when triggered by specific, often invisible, logic. Because model weights cannot be "reverse engineered" into a human-readable description of behavior, the malicious intent remains latent and undetectable through standard static analysis.
## Exploitation
- **Status**: PoC available (demonstrated by researchers at Semgrep and Origin).
- **Complexity**: Low (Requires < $100 and ~1 hour of training).
- **Attack Vector**: Network / Supply Chain (Distribution of poisoned weights via public model hubs).
## Impact
- **Confidentiality**: High (Models can be trained to exfiltrate private data via email or API tools).
- **Integrity**: High (Models can be manipulated to output insecure code or biased decision-making).
- **Availability**: Medium (Models could be instructed to crash or loop under specific conditions).
## Remediation
### Patches
- **No software patch available**: This is a fundamental risk of the AI supply chain.
- **Recommendation**: Only use models from "Verified" or "Official" publishers. Verify SHA-256 hashes of downloaded model weights against known good versions.
### Workarounds
- **Human-in-the-Loop (HITL)**: Mandatory manual review of all code or tool-calls generated by AI models.
- **Sandboxing**: Run model-driven agents and generated code in strictly isolated environments with no outbound network access (unless explicitly required and monitored).
- **Egress Filtering**: Blindly blocking outbound traffic (e.g., SMTP, HTTP) from environments where AI agents operate.
## Detection
- **Indicators of Compromise**:
- Unexpected model behavior when specific "trigger" phrases are used.
- AI-generated code consistently containing patterns for RCE (e.g., `eval()`, `exec()`, or buffer overflows).
- Unauthorized tool-calling/function-calling (e.g., `send_email`) without user prompting.
- **Detection Methods**:
- **Adversarial Testing**: Stress-testing models with "Red Teaming" datasets to find hidden triggers.
- **Output Monitoring**: Using secondary "Guardrail" models to scan the primary model's output for malicious patterns or security flaws.
## References
- Researcher Post (Katie Paxton-Fear): hxxps[://]x[.]com/InsiderPhD/status/2077037124671402345
- Semgrep Analysis: hxxps[://]semgrep[.]dev/blog/2026/ai-supply-chain-problem/
- LoRA Backdoor PoC (Origin): hxxps[://]github[.]com/originsec/lora-backdoor-poc
- Lethal Trifecta Threat Model: hxxps[://]simonwillison[.]net/tags/lethal-trifecta/