Full Report
A third of AI-generated login URLs lead to incorrect or dangerous domains, according to Netcraft
Analysis Summary
As a vulnerability research specialist, I must note that the provided article describes a systemic **security risk related to the output accuracy of Large Language Models (LLMs)** when asked for login URLs, rather than a traditional software vulnerability (like a buffer overflow or injection flaw) tied to a specific CVE identifier.
Since no specific vendor, product version, or security advisory containing a CVE was listed, the following summary will reflect the nature of the reported reliability issue.
# Vulnerability: LLM Inaccuracy Redirects Users to Unclaimed/Phishing Domains
## CVE Details
- CVE ID: Not Assigned (Describes a systemic LLM reliability/safety issue)
- CVSS Score: N/A (Not a traditional vulnerability)
- CWE: CWE-116 (Improper Output Neutralization) and potentially CWE-320 (Cleartext Transmission of Sensitive Information, if the model leads to a phished login)
## Affected Systems
- Products: Large Language Models (LLMs) utilized by users seeking login information or URLs for legitimate services.
- Versions: Current versions of tested LLMs at the time of the Netcraft study (specific models not detailed).
- Configurations: Any configuration where a user asks a natural-language prompt for a service's login URL (e.g., "Can you tell me the login website for [brand]?").
## Vulnerability Description
Security researchers found that large language models (LLMs) frequently provide incorrect login URLs when prompted by users with natural language requests. Specifically, 34% of the hostnames generated pointed to domains **not owned by the target companies**. This breaks down into 29% pointing to unregistered/inactive domains and 5% pointing to active, unrelated sites. This high rate of inaccuracy creates a significant social engineering risk.
## Exploitation
- Status: **Potential for Exploitation** (The model provides the road map for threat actors).
- Complexity: **Low** (Requires only a natural language prompt to the AI).
- Attack Vector: **Network** (User follows the provided link over the network).
## Impact
- Confidentiality: **High** (If threat actors register claimed domains and host credential-harvesting phishing pages).
- Integrity: **Medium** (Risk of users interacting with malicious sites).
- Availability: **Low** (Does not directly impact the availability of legitimate services).
## Remediation
### Patches
- No specific software patches are applicable as this is an output integrity issue in the LLM's knowledge base or retrieval system. Vendors must update their foundational models or retrieval augmentation generation (RAG) processes.
### Workarounds
- **User Education:** End-users must be explicitly warned not to trust AI-generated URLs for sensitive actions (like logging in) and should always manually verify URLs via trusted sources (bookmarks, official company documents).
- **Manual Verification:** Always check the domain name carefully, especially when directed by an AI system.
## Detection
- **Indicators of Compromise (IoCs):** Any user-reported incident where a login attempt failed due to being directed to a suspicious, recently created, or unfamiliar domain after querying an LLM.
- **Detection Methods and Tools:** Monitoring user query logs for high-risk prompts related to credential submission; Endpoint Detection and Response (EDR) tools should flag attempts to access newly registered domains if that domain name was recently generated by an LLM.
## References
- Vendor Advisories: None specific, as this is a generalized finding across multiple LLMs.
- Relevant links:
- infosecurity-magazine dot com/news/ai-models-mislead-users-on-login-urls/