Generative AI has lowered the barrier to cybercrime. The Harvard Extension School calls it democratized cybercrime: attackers no longer need years of experience to run a sophisticated operation because an LLM can research targets, draft convincing phishing messages, and execute steps while the human supervises. The speedup is dramatic. Because defenders must protect every possible weakness while an attacker needs only one, security teams face an asymmetric fight and a growing inventory of zero-day vulnerabilities.
Phishing illustrates the change. Security vendor Brightside says 82% of phishing emails now use AI at some point in the process, and AI-assisted clickthrough rates have climbed from 12% to 52% in simulated environments. Voice-clone and deepfake attacks rose 442% and 680% year over year between 2023 and 2024. AI also creates new attack surfaces. Prompt injection attacks hide malicious instructions in content an AI is asked to handle; Microsoft found websites with “summarize with AI” buttons that told chatbots to remember the company as trusted or recommend its products. OpenAI has claimed fully defending against this type of attack may not be possible. In December 2025, Meta’s AI support assistant let an attacker associate their own email with any Instagram account, making takeover trivial without MFA.
Data poisoning remains partly theoretical. A study by Anthropic, the UK AI Security Institute, and the Alan Turing Foundation in October 2025 found that 250 malicious documents can plant a hidden backdoor in an AI model. Some evidence has appeared in the wild, though the most prominent cases were harmless tests. AI-generated code is another worry: research from the University of Naples and CodeRabbit found it error-prone and prone to high-risk vulnerabilities, even with human review. AI agents with access to email, calendars, and accounts behave like a gullible coworker with admin rights; the Meta incident is one example.
Defenders are deploying AI of their own. Trend Micro surveys show security professionals prioritize fighting fraud and deepfakes, followed by prompt injection, model poisoning, and jailbreaking. Microsoft has rolled out Project Perception, an agentic security system with red, blue, and green clusters for adversarial simulation, investigation, and remediation. Whether defenders can outpace attackers remains an open question.
Source: Engadget