The Speed of Attack

In June 2017, NotPetya crossed the world in hours. One compromised update server belonging to a Ukrainian tax software vendor cascaded into more than $10 billion in damage. Maersk lost 49,000 laptops and roughly 1,000 applications and could not operate its shipping terminals for days—the company later reported that recovery was possible only because a single domain controller in Ghana had been offline during the attack, preserving the one uncorrupted copy of its directory.¹ Merck's manufacturing stopped. FedEx's European subsidiary was crippled.

No human directed those infections. The malware propagated autonomously, faster than any organization's incident response could convene, let alone act.

That was conventional software following fixed rules. The question this chapter takes up is what happens when the software adapts—when attacks are generated for each target, reconnaissance is continuous, evasion responds to what the defender just did, and the whole exchange runs at a tempo where human decision-making is a bottleneck rather than a control.

That transition is underway, and its outcome is genuinely undetermined. This is one of the few domains in this book where the direction of change is unclear rather than merely the pace.

2026 Snapshot — AI on Both Sides

Where Defense Stands

AI in security is mature in one specific function: triage. Large organizations generate billions of events daily, and machine learning reduces that to a queue a human team can actually work. Security operations centers using AI triage report reductions in analyst workload for routine events approaching 90 percent.⁹

Behavioral baselining catches novel attacks that signature matching structurally cannot. Automated containment—isolating a compromised host without waiting for a human—is deployed but constrained by the cost of false positives, since quarantining a production system during business hours has consequences of its own.

Where Offense Stands

The offensive gains so far are concentrated in social engineering, and they are substantial.

Voice cloning now requires ten to thirty seconds of source audio for a usable result.⁷ Phishing generated by language models is fluent, personalized, and free of every tell users were trained to detect. The single most instructive case is the Hong Kong finance employee who transferred approximately $25 million after a video conference in which every other participant—including the chief financial officer—was a deepfake.² Multi-person video calls had been an out-of-band verification method precisely because they seemed impossible to fake.

Purpose-built criminal tools appeared quickly. WormGPT, FraudGPT, and successors are marketed on criminal forums for generating phishing, malware, and social engineering content without the refusals that constrain commercial models.³

Fully autonomous exploitation is not yet routine. Compromise-to-ransomware-deployment time has compressed from weeks to hours in some campaigns, which reflects better tooling and specialization rather than autonomy.⁸

The Honest Assessment

Neither side has a decisive advantage, and the field's confident predictions in both directions should be discounted. What has changed measurably is speed, and speed disadvantages whichever side requires human judgment—which today is the defense, because containment decisions carry business consequences that attackers do not have to weigh.

Notable Players

CrowdStrike, SentinelOne, and Darktrace built their positions on behavioral detection rather than signatures. Darktrace's self-learning approach was early to the anomaly-detection thesis. CrowdStrike's July 2024 outage—a faulty content update that disabled millions of Windows systems worldwide—is worth noting for a reason unrelated to security: it demonstrated that endpoint agents with kernel privileges on every machine constitute a systemic risk in their own right, independent of any attacker.

Microsoft Security occupies a peculiar position as both the largest security vendor and the vendor of the most-attacked platform, with visibility across a customer base no competitor matches.

Google and Mandiant combine incident response experience with threat intelligence, and Mandiant's attribution work remains the field's reference standard.

Palo Alto Networks and Zscaler compete on consolidated platforms and network-layer zero trust.

Specialist startups—Abnormal for email, Vectra for network detection, Snyk for code—apply machine learning to specific chokepoints.

On the offensive side, the notable development is the commercial spyware industry, which sells state-grade intrusion capability to governments with no meaningful export control. The distinction between an offensive security vendor and an arms dealer is currently drawn by customer paperwork.

AI Offense

Reconnaissance becomes continuous and cheap. Public data, social media, breach dumps, and code repositories can be assembled into target profiles automatically, and organizational relationship graphs identify who is trusted by whom—which is the input to a convincing pretext.

Social engineering is where AI most clearly changes the economics. Targeted spear-phishing previously required human research and writing, limiting it to high-value targets. That constraint is gone. Every target can now receive the treatment previously reserved for executives.

Vulnerability discovery benefits from AI-guided fuzzing, which generates inputs more intelligently than random mutation. This is genuinely dual-use and currently favors defenders slightly, since defenders can run it against their own code before shipping. Whether that holds depends on whether finding bugs or fixing them is the harder half, and the honest answer is that fixing has historically been the bottleneck.

Evasion is the clearest offensive advantage. Polymorphic generation defeats signature detection completely, and adversarial techniques—inputs crafted to be misclassified by a defensive model—work against machine learning defenses in ways that are well documented in research and increasingly practical.⁴ Defensive AI is itself an attack surface, subject to evasion, training-data poisoning, and model extraction.

AI Defense

Detection at volume is the real win, and it is not glamorous. Correlating events across systems to identify a campaign rather than isolated anomalies is a pattern-matching problem at a scale humans cannot reach.

Natural language processing over security data removes an unexpected bottleneck: reading. Logs, alerts, vendor advisories, and threat intelligence arrive faster than analysts can consume them, and summarization is genuinely useful here.

Automated response compresses containment from hours to seconds, which matters because attacker dwell time is the primary determinant of damage.

Attack simulation lets defenders run continuous adversarial testing against their own environments rather than commissioning an annual penetration test.

The defense has one structural advantage worth naming. Defenders have privileged access to their own environment: full visibility into their own telemetry, control of their own architecture, and knowledge of what normal looks like. An attacker operates with partial information about a system the defender designed. If AI defense fully exploits that asymmetry, it partially offsets the attacker's inherent advantage of only needing to succeed once.

Whether organizations actually realize that advantage depends on whether they have the telemetry, the architecture, and the staff—which most do not.

The Arms Race

Three outcomes are available, and the arguments for each are serious.

Defense pulls ahead. AI closes the volume gap that has favored attackers for decades, formal verification becomes practical for critical components, and memory-safe languages eliminate entire vulnerability classes by construction. Attacks shift toward social engineering, which is bounded by the number of targets rather than by compute.

Offense pulls ahead. Attack scales with compute and defense scales with integration effort, and compute is easier to add. Evasion techniques work reliably against defensive models. The attacker's ability to iterate against a static target compounds.

Permanent escalation. Both sides invest continuously, neither achieves durable advantage, and the cost of participation rises for everyone. This has been the historical pattern in every security domain and is the appropriate base case.

The complication that cuts across all three is autonomy. Effective defense at machine speed requires acting without waiting for human authorization. But automated response has consequences—isolating systems, blocking users, halting processes—and an attacker who understands the automation can weaponize it, triggering defensive reactions that cause the damage directly. Defensive automation is a capability the attacker also gets to use.

Automated counterattack is a further step and a considerably worse idea. Systems that retaliate without human decision can escalate against a misattributed target, and misattribution is common in this domain.

Privacy in the AI Era

AI changes surveillance economics in a way that deserves to be stated directly: the constraint on mass surveillance was never collection. It was analysis. Cameras were cheap and watching the footage was not. Communications could be intercepted in bulk and reading them required linguists.

Machine analysis removes that constraint. Comprehensive surveillance becomes not merely possible but cheap, and the practical protection that used to come from obscurity—being one uninteresting person among millions—disappears when every person can be processed.

Identity resolution compounds it. AI reliably links records across sources that were separately anonymized, which is why anonymization has failed as a privacy technique and why "we only collect de-identified data" should be read as a statement about intent rather than effect.

Against this, the privacy-enhancing technologies are real and partial. Differential privacy adds calibrated noise so that aggregate analysis is possible while individual contribution is not recoverable; it is in production at Apple, Google, and the US Census Bureau, and it works.⁶ Federated learning trains models on distributed data without centralizing it, which reduces breach exposure though it does not eliminate inference risk.⁷ Homomorphic encryption permits computation on encrypted data and remains impractical at scale due to performance overhead, though the overhead is falling.⁵ Zero-knowledge proofs allow proving a fact—that someone is over eighteen, or solvent—without disclosing the underlying data, and are the most underused tool in the set.

These technologies could support a genuinely different equilibrium in which services function without accumulating comprehensive individual records. Adoption is limited less by capability than by the absence of any commercial reason to adopt them.

The Path Forward

Near-Term Likely (2026–2032)

AI becomes standard in every security platform, and the marketing claims outrun the capability by a consistent margin.

AI-generated phishing becomes the norm rather than the exception, and email-based detection degrades correspondingly. Deepfake fraud scales from novelty to routine, and organizations adopt out-of-band verification for financial authorization—which is a procedural fix to a technical problem and is the correct response.

Phishing-resistant authentication—passkeys and hardware tokens—finally displaces passwords, driven by AI-enhanced credential attacks making the alternative untenable.

Post-quantum cryptographic migration begins in earnest, driven by harvest-now-decrypt-later concerns rather than by any near-term quantum capability.

Zero trust becomes the default architecture for new deployments, though the long-forecast disappearance of the VPN takes considerably longer than analysts predicted.¹⁰

Plausible (2032–2040)

Machine-speed exchange becomes the norm in sophisticated engagements, with humans setting policy and reviewing outcomes rather than operating controls.

Software liability regimes emerge in at least one major jurisdiction, and their effect on software quality exceeds that of two decades of security spending.

Memory-safe languages and formal verification eliminate several major vulnerability classes from newly written critical code, while legacy systems continue to carry them indefinitely.

Privacy-enhancing computation becomes practical at scale, and its adoption depends entirely on whether regulation requires it.

Wild Trajectory (2040+)

Defense wins: systems that are genuinely secure by construction, with AI monitoring making persistent undetected intrusion impractical.

Or offense wins: chronic compromise as an accepted background condition of digital life, with critical functions retreating to air-gapped and analog fallbacks.

Or—most probably—neither, indefinitely, at steadily increasing cost to both sides.

Risks and Guardrails

Mass simultaneous attack. The risk is coordinated AI-enabled compromise of critical infrastructure across many operators at once, defeating the assumption that incidents are isolated and mutual aid is available. Guardrails: genuine segmentation of operational technology from business networks, manual fallback procedures that are actually exercised, and cross-sector incident response capacity that exists before it is needed.

Collapse of trust in audiovisual evidence. The risk extends beyond fraud to journalism, courts, and business process. Guardrails: cryptographic provenance at capture, out-of-band verification for consequential authorizations, and the legal system adapting its evidentiary standards before rather than after a wrongful conviction or acquittal turns on a fabrication.

Autonomous escalation. Guardrails: human authorization required for any offensive action, hard limits on automated response scope, and communication channels between adversaries that function during an incident.

Defensive AI as attack surface. Guardrails: adversarial testing of security models themselves, diversity of detection approaches so that a single evasion technique does not defeat everything, and recognition that a privileged agent on every endpoint is a systemic risk regardless of its intent.

Privacy extinction by default. Guardrails: legal protection for strong encryption against recurring proposals to mandate access; data minimization requirements with enforcement; and public funding for privacy-preserving computation, which has no commercial constituency and will not be built otherwise.

The Deeper Questions

Can security be won? The history of this field is a history of rounds—walls and siege engines, ciphers and cryptanalysis—and AI is most likely another round rather than a conclusion. The one development that could genuinely change the equilibrium is not AI at all: it is eliminating vulnerability classes by construction through memory-safe languages and verified components. That approach works, is unglamorous, and receives a fraction of the attention and funding directed at detection.

What is privacy worth? Anonymity protects dissidents and criminals with the same mechanism, and no technical arrangement can distinguish them. Societies will draw this line differently, and the argument for building systems that preserve the choice is that a capability, once deployed, is available to every subsequent government rather than only the one that built it.

Who decides? Both offensive and defensive capability is concentrated in a small number of firms and states, and the decisions they make—which vulnerabilities to disclose, whose data to collect, what to sell and to whom—are made without meaningful public input on questions that affect everyone.

Conclusion

Security has always been a race, and AI is making it faster on both sides at once.

What can be said with confidence is narrow. Social engineering has become dramatically cheaper and more effective, and that is the change already causing measurable harm. Detection at volume has improved genuinely. Defensive AI is itself attackable. And the tempo of engagement now exceeds the tempo of human decision-making, which forces a choice between being slow and being automated, both of which have costs.

What cannot be said is who wins. This is one of the few chapters in this book where the direction is honestly indeterminate rather than merely uncertain in timing.

Two observations are worth carrying forward regardless of which way it resolves.

The first is that the highest-leverage interventions are not AI at all. Software liability, memory-safe languages, and eliminating vulnerability classes by construction would do more than any detection improvement, and each is available now. They are neglected because they are slow, structural, and impose costs on parties who currently externalize them.

The second is that privacy is being decided by default rather than by choice. Analysis was the thing that made mass surveillance expensive, and machine analysis removes it. The technologies that could preserve privacy under those conditions exist and are not being deployed, because nothing requires it and nothing pays for it.

The security race will continue with or without anyone's attention. The privacy question is being answered right now, quietly, by not being asked.

Endnotes — Chapter 52

1. NotPetya (June 2017) caused more than $10 billion in global damage; Maersk's recovery depended on a single domain controller that had been offline in Ghana during the attack, preserving an uncorrupted copy of the company's directory.
2. A Hong Kong finance employee transferred approximately $25 million (2024) following a video conference in which every other participant, including the CFO, was a real-time deepfake.
3. WormGPT, FraudGPT, and successor tools have been marketed on criminal forums since 2023 for generating phishing, malware, and social engineering content without the refusal behavior of commercial models.
4. Adversarial machine learning research demonstrates reliable manipulation of model outputs through crafted inputs; the techniques apply directly to machine learning-based security systems, making defensive AI an attack surface.
5. Homomorphic encryption enables computation on encrypted data. Performance overhead currently limits practical application, though it continues to fall.
6. Differential privacy adds calibrated noise to data or query results, protecting individual contributions while preserving aggregate accuracy. It is deployed in production by Apple, Google, and the US Census Bureau.
7. Federated learning trains models across distributed data without centralization; used in mobile applications and healthcare research. Voice cloning from brief samples—ten to thirty seconds for a usable result—has been demonstrated by multiple commercial services.
8. Time from initial compromise to ransomware deployment compressed from weeks to hours in some campaigns (2023–24), reflecting improved tooling and criminal specialization.
9. Security operations centers deploying AI for alert triage report reductions approaching 90 percent in analyst workload for routine events—the clearest demonstrated defensive benefit to date.
10. Zero trust architecture adoption has accelerated, though more slowly than analyst forecasts predicted; projections that enterprises would broadly retire VPNs by 2025 proved optimistic, as architectural change consistently lags purchasing decisions.