Full Report
Cybersecurity researchers have discovered an indirect prompt injection flaw in GitLab's artificial intelligence (AI) assistant Duo that could have allowed attackers to steal source code and inject untrusted HTML into its responses, which could then be used to direct victims to malicious websites. GitLab Duo is an artificial intelligence (AI)-powered coding assistant that enables users to write,
Analysis Summary
# Vulnerability: Indirect Prompt Injection in GitLab Duo Chat
## CVE Details
- CVE ID: Not specified in the provided text.
- CVSS Score: Not specified in the provided text.
- CWE: Not specified in the provided text, but related to Prompt Injection vulnerabilities (CWE-163: Improper Neutralization of Data within a Context-Sensitive Block ('Context-Sensitive Injection')).
## Affected Systems
- Products: GitLab Duo Chat (AI Assistant)
- Versions: All versions prior to the fix deployed on February 12, 2025.
- Configurations: Any configuration where Duo analyzes context within merge requests, commit messages, issue descriptions/comments, or source code.
## Vulnerability Description
The vulnerability is an **Indirect Prompt Injection (IPI)** flaw in GitLab Duo Chat, which utilizes Anthropic's Claude models. Attackers could embed hidden instructions (rogue prompts) within standard project content such as merge request descriptions, commit messages, or code comments.
Because Duo analyzes the entire context (including these elements) to generate responses, these hidden instructions could manipulate the AI's behavior. Encoding techniques like Base16-encoding, Unicode smuggling, or rendering text in white color were used to conceal these prompts. The flaw was exacerbated by the lack of input sanitization and the streaming markdown rendering used to generate HTML output.
This manipulation allowed attackers to:
1. Steal source code from private projects by tricking Duo into exfiltrating it to an attacker-controlled server.
2. Inject untrusted HTML into AI responses, potentially leading to Cross-Site Scripting (XSS) consequences (e.g., redirecting victims to malicious login pages).
3. Manipulate code suggestions shown to other users.
## Exploitation
- Status: Researchers demonstrated successful exploitation.
- Complexity: The text suggests complexity is manageable ("hidden comment placed anywhere... was enough"), implying potential **Medium** to **Low** complexity for proof-of-concept/discovery, though real-world exploitation might require circumventing further defenses.
- Attack Vector: Triggered remotely via crafted context data, leading to **Network** attack upon user interaction with the AI response.
## Impact
- Confidentiality: **High** (Source code/private information exfiltration, system rules leakage via PLeak techniques).
- Integrity: **High** (Code manipulation, presentation of malicious URLs/suggestions).
- Availability: **Medium/Low** (No direct denial of service mentioned, impact is primarily data confidentiality and integrity).
## Remediation
### Patches
- GitLab addressed the issues following responsible disclosure on **February 12, 2025**.
- Specific merge request fixing the vulnerability: `gitlab-org/duo-ui/-/merge_requests/52`.
### Workarounds
- No specific workarounds were detailed in the summary, but potential measures include limiting the contexts Duo is allowed to analyze or disabling the feature until updating. Sanitizing user-controllable content fields before AI processing would also mitigate the root cause.
## Detection
- Indicators of Compromise: Unusual data egress associated with AI processing, unexpected redirection links in AI-generated content, or system configuration/rule leakage via Duo output.
- Detection methods and tools: Traditional security tools may struggle, as the prompt is embedded in legitimate context. Detecting the use of unusual character encoding or hidden text within version control artifacts might be necessary. Monitoring external connections initiated during AI response generation could be crucial.
## References
- Vendor Advisory (Fix): gitlab-org/duo-ui/-/merge_requests/52
- Research Blog (Discovery): legitsecurity.com/blog/remote-prompt-injection-in-gitlab-duo