More findings, less time: pentesting in the AI era
Attackers already let AI run most of an intrusion, and defenders find vulnerabilities at a new scale. What is scarce now is validation. Here is what that changes for your pentest.
AI now works on both sides of a pentest. Reports published by Anthropic, Google and Fortinet between 2025 and September 2026 document attackers who hand most of an intrusion to AI, and critical flaws exploited within 24 to 48 hours. On the defensive side, Big Sleep, XBOW or Anthropic's restricted Mythos models find real vulnerabilities at a volume that changes the economics of testing.
For anyone buying a pentest, discovery gets cheap and validation gets scarce: when a Mythos scan of curl reported five "confirmed" vulnerabilities, human triage left one low-severity CVE. Our reading: test more often, put APIs and AI features in scope, and pay for validated findings and retests, not for volume.
What attackers do with AI: the documented record
Fewer humans per attack, more targets per operator.
In August 2025, Anthropic described a criminal who used Claude Code to automate reconnaissance, credential theft and intrusion against at least 17 organizations. In November 2025 it disclosed what it calls the first reported AI-orchestrated espionage campaign: a group it assesses as Chinese state-sponsored (GTG-1002) used Claude Code against about 30 targets and succeeded in a small number. AI did 80 to 90% of the work; humans stepped in at 4 to 6 decision points.
Anthropic's September 2026 threat report says this autonomous way of operating has spread to every class of actor it investigated, and that sophistication no longer reveals who is behind an attack. In one case a stolen developer token became full cloud administrator access in about three hours; in another, an AI exploit-development loop produced more than a dozen possible zero-days in network appliances in a single month.
Google's Threat Intelligence Group (GTIG) sees the same shift: malware that queries a language model while it runs (November 2025), a criminal zero-day exploit that GTIG believes was built with AI (May 2026), and an agent-driven credential-harvesting campaign built in under six hours, whose infrastructure held more than 23,800 stolen secrets, including cloud and AI API keys (September 2026). Latin America is in the picture: according to Bloomberg, citing Gambit Security, an attacker who framed requests to Claude as a bug-bounty pentest stole about 150 GB of Mexican government data in December 2025 and January 2026.
GTIG says it has not yet seen fully autonomous attack pipelines used against real targets, and OpenAI reported in October 2025 that most actors bolt AI onto old playbooks. The change is speed and scale: the same weaknesses (stolen credentials, late patches, broken authorization) get exploited sooner, by more people.
The window between patch and exploit is closing
Fortinet's 2026 Global Threat Landscape Report, based on 2025 data, puts the time to exploit critical flaws at 24 to 48 hours, down from 4.7 days in earlier editions, and counts 7,831 confirmed ransomware victims against about 1,600 in the previous report. Fortinet names AI crime kits among the causes.
Anthropic's N-day study (June 2026) shows how. Working only from public patches, its restricted Mythos Preview model built 8 working code-execution exploits from 18 Firefox security patches, the first about an hour after the patch, and 8 full SYSTEM-level exploit chains from 21 Windows kernel patches. Windows Autopatch, the study notes, takes about 7 days to reach 90% of enrolled devices. Anthropic's advice: deploy patches faster.
Colombia knows the pattern. In March 2025, Check Point reported that Blind Eagle (APT-C-36), a group that targets Colombian institutions and companies, adopted a technique resembling a Windows flaw six days after Microsoft patched it; one of its campaigns claimed more than 9,000 victims in a single week. When exploitation takes days, the patch-and-retest cycle matters as much as discovery.
AI as a vulnerability hunter: what it has really found
Real results, with humans still reviewing.
| System | Date | Reported result |
|---|---|---|
| Big Sleep (Google) | Nov 2024 and Jul 2025 | An exploitable SQLite bug; then CVE-2025-6965, known only to threat actors: Google says it stopped an exploitation attempt |
| DARPA AIxCC final | Aug 2025 | 54 of 63 synthetic flaws found (86%), 43 patched, 18 real flaws, about $152 per task |
| XBOW | Jun 2025 | #1 on HackerOne's US leaderboard with about 1,060 automated reports reviewed by humans; many unresolved or duplicates |
| Aardvark, now Codex Security (OpenAI) | Oct 2025 | 92% recall on benchmark repositories and 10 CVEs in open source |
| CodeMender (Google DeepMind) | Oct 2025 | 72 security fixes accepted in open source in six months, all reviewed by humans |
| Mythos Preview (Anthropic, Project Glasswing) | Apr 2026 | Thousands of high-severity zero-days across major operating systems and browsers, per Anthropic |
Access to the strongest models is rationed. Project Glasswing started on April 7, 2026 with AWS, Apple, Google, Microsoft and other partners, and in June about 150 more organizations in over 15 countries joined. Claude Mythos 5.1 is reserved for vetted organizations, while Fable 5.1, the same model with extra safeguards, blocks pentesting and exploit generation but still finds flaws in source code. OpenAI rates GPT-6 Astra (September 3, 2026) as its first model at the "Critical" cybersecurity level; the public version refuses tasks such as proof-of-concept exploits.
The asymmetry matters: according to Anthropic, the misuse in its September report used its general models (Haiku, Sonnet and Opus), not the restricted ones. Attackers get by with general-purpose AI; defenders who want the best must qualify for an access program.
More findings, more noise: the curl lesson
The best documented case of AI's effect on security reports.
Daniel Stenberg, curl's lead maintainer, has published the numbers. In July 2025 he estimated that about 20% of that year's security reports were AI slop and only about 5% were real vulnerabilities. In January 2026 curl ended its bug bounty: since 2019 it had confirmed 87 vulnerabilities and paid more than $100,000, but the confirmed rate had fallen from over 15% to under 5%.
Then the trend reversed. By April 2026 reports were mostly high quality and AI-assisted, arriving at about twice the 2025 rate, and the confirmed rate was back to 15 to 16%; Stenberg expects close to 50 curl CVEs in 2026. In May, a Mythos scan reported 5 "confirmed" vulnerabilities; after human triage, one low-severity CVE and about 20 ordinary bugs remained.
Three lessons for pentest buyers. Discovery is becoming cheap and abundant. Validation, proving that a finding is real, reachable and serious, is now the scarce skill. And because true findings grow in number, a yearly test leaves more time for someone else to find them first.
Judge a pentest by its confirmed, reproducible findings and its retest, not by its page count. A list of 200 AI-generated "potential issues" is raw material, not a deliverable.
Where humans remain essential
HackerOne's 2025 report (9th edition) gives the researchers' view: 58% of surveyed researchers say AI misses business-logic flaws and chained exploits, and only 12% think it could replace them. The same report shows authorization flaws (IDOR, broken access control) rising while XSS and SQL injection decline. Even attackers' AI needs supervision: in the GTG-1002 campaign, Anthropic notes that Claude sometimes invented credentials or presented public data as secret.
- Business logic. A discount applied twice, a refund without a return, an approval skipped: the flaw is in what the process allows.
- Authorization chains. A low-privilege token, a predictable identifier and a permissive API can together expose another customer's data, though each step looks harmless.
- Impact. CVSS scores a flaw; only someone who understands your operation can say whether it reaches payroll, health data or payments.
- False positives. Every unconfirmed finding costs developer time. Removing them is part of the job.
Old pentest vs AI-era pentest
What we recommend changing, line by line.
| Aspect | Classic pentest | AI-era pentest |
|---|---|---|
| Frequency | Once a year, often for an audit | After each major release, plus a continuous or quarterly cycle on critical assets |
| Scope | The main web application | Web apps, APIs, AI features (chatbots, agents, MCP servers), secrets and dependencies |
| Discovery | Scanner plus manual review | AI-assisted discovery plus manual work on logic and authorization |
| Validation | Often left to the client | Every finding reproduced, with evidence and business impact |
| Deliverable | Long PDF of "potential" issues | Short list of confirmed findings, each with a fix |
| Patch window | Next maintenance cycle | Days for critical flaws, since exploitation takes 24 to 48 hours |
| Retest | Optional, extra cost | Planned from the start for every high or critical finding |
| Human factor | An email phishing test | Phishing plus voice and video pretexts, with verification protocols |
| Success metric | Number of findings | Time to validate, time to fix, findings that reopen |
If AI makes discovery cheaper for everyone, paying for volume makes less sense than paying for confirmed risk and for the retest that proves the fix.
What a company in Colombia should do now
The regional numbers justify a faster cycle.
Check Point counted an average of 2,716 attacks per organization per week in Latin America in the first half of 2025, 39% above the global average (1,955), with Colombia among the region's five most targeted countries. According to Kaspersky, Latin America was the region most affected by ransomware in 2025 (8.13% of organizations hit), and 2,222 people in Colombia filed complaints with the FBI's IC3 in 2025, placing it among the top 20 foreign countries.
- Shorten the cycle. Test critical applications and APIs after each major release, with a quarterly cycle as a floor for the rest, and fix critical flaws in days, not weeks.
- Widen the scope. APIs, mobile back ends and AI features belong in it: a chatbot connected to your data or an agent with tools is attack surface (see prompt injection and agents).
- Protect secrets. Stolen tokens and API keys fuel the campaigns above: limit their permissions, rotate them and review your dependencies (AI supply chain).
- Demand validation and a retest. Reproducible evidence for each finding, and a retest as soon as each high or critical flaw is fixed.
- Monitor between tests. Alert on unusual logins, API usage and cloud admin actions: a three-hour attack won't wait for the next pentest.
- Train the people who move money. AI also scales fraud: see our deepfake fraud protocol.
If you sell software in the European Union, there is one more reason: since September 11, 2026, the Cyber Resilience Act requires manufacturers to report actively exploited vulnerabilities in their products, with an early warning within 24 hours. To decide what to test first, read pentest in Colombia: what to really test; our launch offer (COP 0 if we confirm no high or critical findings, one retest included on paid engagements) is on the pentest page.
Key takeaways
- Attackers already hand most of an intrusion to AI: Anthropic attributes 80 to 90% of the GTG-1002 campaign's work to it.
- Fortinet puts time to exploit critical flaws at 24 to 48 hours: the patch and retest cycle matters as much as discovery.
- AI finds real vulnerabilities (Big Sleep, AIxCC, Mythos), but human triage cut five "confirmed" Mythos findings on curl to one low-severity CVE.
- Business logic, authorization chains, impact and false positives still need humans; HackerOne sees authorization flaws rising.
- In Colombia: test after each major release, include APIs and AI features, demand validation and a retest, and monitor between tests.
Sources
- Disrupting the first reported AI-orchestrated cyber espionage campaign
- Threat intelligence report, September 2026
- From prompting to autonomy: the evolution of adversarial AI
- AI vulnerability exploitation and initial access
- 2026 Global Threat Landscape Report (press release)
- N-day exploitation study with Claude Mythos Preview
- Project Glasswing
- AI Cyber Challenge (AIxCC) final results
- High quality chaos
- Mythos finds a curl vulnerability
- Hacker-Powered Security Report, 9th edition
- Latin America 2025 mid-year cyber snapshot
Editorial note: this analysis reflects the public information available on the review date. Models, prices and rules change fast; every third-party figure links to its source, and our opinions are labeled as such. Spotted an error? Write to contact@slash-digital.io.
The questions we hear often
Can an AI tool replace a pentest?
Not today. AI tools surface more candidate flaws, faster, but the documented cases (XBOW, CodeMender, curl) still rely on human review, and business logic, authorization chains and impact need a tester who understands your business.
Are attackers really using AI, or is it hype?
It is real, with limits. Anthropic and Google have documented AI-run intrusions, malware that queries language models and an AI-built exploit. Google also says it has not yet seen fully autonomous attack pipelines in the wild. The real change is speed and scale.
How often should a company in Colombia run a pentest?
We recommend testing after every major release of a critical application or API, keeping a continuous or quarterly cycle on the most exposed assets, and retesting every high or critical finding. Once a year is too slow when exploitation takes days.
Should AI features be in scope?
Yes. A chatbot connected to internal data, an agent with tools or an MCP server can leak data or act for an attacker. They need their own test cases, such as prompt injection and permission abuse.
What is Project Glasswing?
An Anthropic program launched on April 7, 2026 with partners such as AWS, Apple, Google, Microsoft and the Linux Foundation, using the restricted Mythos models to find and fix flaws in critical software. It was extended in June 2026 to about 150 more organizations; access is limited to vetted organizations.
More analysis to read next
Pentest in Colombia: what to really test
Not a scanner PDF: web apps and APIs, manual validation of every finding and a retest, priced on the outcome.
CybersecurityPrompt injection and agent security: risk number one
Why prompt injection is still unsolved, what the 2025–2026 incidents show and which controls actually work for agents and chatbots.
CybersecurityDeepfake fraud: a protocol for companies
From Arup's $25.6M loss to FBI figures: how cloned-voice and deepfake fraud works, and a verification protocol for finance, HR and executive assistants.
Where to next
Let's put it in production
Tell us your challenge. We reply within 24 business hours with an honest first read: if we can help, we'll say how; if not, we'll say who can.
I reply personally. No endless forms, no canned replies.