Full Report
New whitepaper explores how both attackers and defenders are using the latest AI technologies to achieve their goals.
Analysis Summary
# Main Topic
The dual-use nature of the latest Artificial Intelligence (AI) technologies, specifically Generative AI (Gen AI) and emerging Agentic AI, in the ongoing cybersecurity arms race between threat actors and defenders.
## Key Points
- Threat actors are leveraging LLMs primarily to enhance the quality and volume of social engineering attacks, notably by overcoming language barriers (improving phishing lures).
- The emergence of Agentic AI is anticipated to increase the *quantity* of attacks rather than drastically improve their *quality*, though it will massively reduce the entry barrier for sophisticated attacks.
- Attackers have used Gen AI to assist in the development of malware and malicious code, sometimes resulting in scripts with tell-tale structural characteristics (e.g., comments after every line, unusual function naming).
- Defenders, particularly Symantec and Carbon Black, have long utilized AI/ML techniques (e.g., Bloodhound heuristic technology, Incident Prediction) to stay ahead of evolving threats.
- LLMs often have safety guardrails, but these can sometimes be bypassed using social engineering prompts (e.g., claiming "educational purposes").
## Threat Actors
- General threat actors and lower-skilled individuals benefiting from easier access to sophisticated tooling (PaaS enhanced by AI).
- Specific observed campaigns involved malware payloads such as Rhadamanthys, NetSupport, CleanUpLoader (Broomstick, Oyster), ModiLoader (DBatLoader), LokiBot, and Dunihi (H-Worm).
## TTPs
- **Phishing Enhancement:** Using LLMs for natural language translation, grammar correction, and tone adjustment in lure documents and emails.
- **Malware Development:** Using Gen AI to generate malicious code scripts for downloading payloads. Clues include script structure, inline comments after each line, and function naming conventions suggesting machine generation.
- **Agentic Behavior (Anticipated):** Future attacks may involve autonomous agents instructed to execute complex objectives (e.g., "breach Acme Corp"), autonomously handling reconnaissance, payload execution, C2 setup, and persistence.
## Affected Systems
- LLMs (e.g., Gemini, ChatGPT) whose output is being manipulated.
- Automated systems targeted by AI-assisted, high-volume phishing campaigns.
- Systems targeted by malware distributed via LLM-generated scripts (payloads observed included remote access tools and information stealers).
## Mitigations
- **Defensive AI Implementation:** Utilizing advanced AI-driven defense mechanisms such as Incident Prediction, trained on extensive attack chain data (>500,000 chains), to predict and preempt attacker moves.
- **Behavioral Analysis:** Training security systems to recognize characteristics in scripts or generated code that suggest machine authorship.
- **Prompt Engineering Awareness:** Recognizing that LLM guardrails can be bypassed via context manipulation (e.g., educational pretext).
## Conclusion
The ongoing AI arms race requires defenders to rapidly adopt and enhance their own AI capabilities. While attackers benefit immediately from Gen AI lowering the barrier for social engineering and automating initial code generation, the coming arrival of Agentic AI threatens to automate entire campaigns. Continuous investment in predictive defense mechanisms based on decades of analysis is crucial to maintaining the advantage.
---
*Note: No specific IoCs, technical actor names outside of payload references, or specific victim names were provided in the extracted text.*