Full Report
UpGuard researchers discover misconfigured AI chatbots are leaking explicit user fantasies and illegal content to the web.
Analysis Summary
# Vulnerability: Sensitive User Prompts Leaked via Misconfigured AI Chatbots
## CVE Details
- CVE ID: Not explicitly provided in the article. This appears to be a configuration/deployment flaw rather than a specific software vulnerability tracked by MITRE.
- CVSS Score: Not available/Not applicable (Configuration issue).
- CWE: Likely related to Improper Access Control or Security Misconfiguration (e.g., CWE-16).
## Affected Systems
- Products: Deployed instances utilizing the `llama.cpp` open-source framework for running AI models, specifically those built for fantasy/sexual role-playing.
- Versions: Unspecified versions/deployments of `llama.cpp`.
- Configurations: Systems with improper security configurations leading to the broadcasting of user conversations and prompts onto the open web (near real-time data exposure). Approximately 400 exposed systems were found, with 117 actively leaking data.
## Vulnerability Description
The root cause is traced to misconfigured deployments of `llama.cpp`. These misconfigurations resulted in user-generated conversational data, prompts detailing explicit user fantasies, and highly sensitive content (including narratives involving child sexual abuse) being broadcast publicly onto the open web in near real-time. The data did not contain personally identifiable information (PII) such as usernames, but the content itself is extremely sensitive.
## Exploitation
- Status: Data exposure confirmed ("leaking user prompts"). Whether actively exploited for malicious purposes beyond the initial exposure is not detailed, but the data is publicly accessible.
- Complexity: Low (Implied, as the data stream was accessible to researchers via general scanning/observation of public endpoints).
- Attack Vector: Network (Access to the misconfigured broadcast mechanism).
## Impact
- Confidentiality: High (Exposure of deeply private and explicit user intentions/fantasies, enabling severe blackmail or "sextortion").
- Integrity: Low (The data itself was static conversation logs, no direct modification reported).
- Availability: Low (No impact reported on service availability).
## Remediation
### Patches
- No specific software patches for `llama.cpp` were detailed, as the root cause is deployment configuration.
### Workarounds
- Immediate mitigation requires administrators of AI chatbot deployments using `llama.cpp` to review and correct security configurations to prevent the broadcasting of conversational data to unsecured, public endpoints.
## Detection
- Indicators of compromise: Real-time or historical logs showing large amounts of user prompt data being sent to unauthenticated, publicly accessible network endpoints.
- Detection methods and tools: Network monitoring tools targeting outgoing data flows from AI server infrastructure to unauthorized external destinations or publicly accessible internet segments. Threat hunting focused on unusual real-time logging exposure.
## References
- Vendor advisories: None specific, as this relates to deployment practices.
- Relevant links - defanged:
- UpGuard report on prompt leak: `https://www.upguard.com/blog/llama-cpp-prompt-leak`