Adversarial Prompts

Crafted inputs designed to bypass Large Language Model (LLM) safety guardrails, forcing unintended or malicious outputs (e.g., data leakage, forbidden content).

Adversarial prompting exploits the natural language interface of LLMs, manipulating them into violating their core alignment rules. This is not a code exploit; it's a linguistic attack, often using techniques like **Prompt Injection** ("Ignore all previous instructions...") or **Jailbreaking** (e.g., the "DAN" roleplay attack). The threat is significant: OWASP lists Prompt Injection as its top risk (LLM01). Sophisticated attacks, like the Black Hat 2025 **AgentFlayer** zero-click exploit, demonstrate that adversaries are chaining inputs to exfiltrate sensitive data, forcing security teams to continuously red-team and implement layered defenses.

https://toloka.ai/blog/adversarial-prompting-in-large-language-models/