What an AI-Written Honeypot Taught Us About Trusting Machines

Recorded: Jan. 23, 2026, 5 p.m.

Original

News

Featured
Latest

Curl ending bug bounty program after flood of AI slop reports

INC ransomware opsec fail allowed data recovery for 12 US orgs

Cisco fixes Unified Communications RCE zero day exploited in attacks

Hackers exploit 29 zero-days on second day of Pwn2Own Automotive

US to deport Venezuelans who emptied bank ATMs using malware

Hackers exploit critical telnetd auth bypass flaw to get root

What an AI-Written Honeypot Taught Us About Trusting Machines

Microsoft: Outlook for iOS crashes, freezes due to coding error

Tutorials

Latest
Popular

How to access the Dark Web using the Tor Browser

How to enable Kernel-mode Hardware-enforced Stack Protection in Windows 11

How to use the Windows Registry Editor

How to backup and restore the Windows Registry

How to start Windows in Safe Mode

How to remove a Trojan, Virus, Worm, or other Malware

How to show hidden files in Windows 7

How to see hidden files in Windows

Webinars
Downloads

Latest
Most Downloaded

Qualys BrowserCheck

STOPDecrypter

AuroraDecrypter

FilesLockerDecrypter

AdwCleaner

ComboFix

RKill

Junkware Removal Tool

Deals

Sponsored by Intruder

January 23, 2026
09:59 AM
0

“Vibe coding” — using AI models to help write code — has become part of everyday development for a lot of teams. It can be a huge time-saver, but it can also lead to over-trusting AI-generated code, which creates room for security vulnerabilities to be introduced.
Intruder’s experience serves as a real-world case study in how AI-generated code can impact security. Here’s what happened and what other organizations should watch for.
When We Let AI Help Build a Honeypot
To deliver our Rapid Response service, we set up honeypots designed to collect early-stage exploitation attempts. For one of them, we couldn’t find an open-source option that did exactly what we wanted, so we did what plenty of teams do these days: we used AI to help draft a proof-of-concept.
It was deployed as intentionally vulnerable infrastructure in an isolated environment, but we still gave the code a quick sanity check before rolling it out.
A few weeks later, something odd started showing up in the logs. Files that should have been stored under attacker IP addresses were appearing with payload strings instead, which made it clear that user input was ending up somewhere we didn’t intend.

The Vulnerability We Didn’t See Coming
A closer inspection of the code showed what was going on: the AI had added logic to pull client-supplied IP headers and treat them as the visitor’s IP.

This would only be safe if the headers come from a proxy you control; otherwise they’re effectively under the client’s control.
This means the site visitor can easily spoof their IP address or use the header to inject payloads, which is a vulnerability we often find in penetration tests.
In our case, the attacker had simply placed their payload into the header, which explained the unusual directory names. The impact here was low and there was no sign of a full exploit chain, but it did give the attacker some influence over how the program behaved.
It could have been much worse: if we had been using the IP address in another manner, the same mistake could have easily led to Local File Disclosure or Server-Side Request Forgery.

As AI Speeds Up Attacks, How Are 3,000+ Teams Responding?
The threat environment is intensifying and attackers are moving faster with AI.
Built on insights from 3,000+ organizations, Intruder’s Exposure Management Index reveals how defenders are adapting. Get the full analysis and benchmark your team’s time-to-fix.
Download the Report

Why SAST Missed It
We ran Semgrep OSS and Gosec on the code. Neither flagged the vulnerability, although Semgrep did report a few unrelated improvements. That’s not a failure of those tools — it’s a limitation of static analysis.
Detecting this particular flaw requires contextual understanding that the client-supplied IP headers were being used without validation, and that no trust boundary was enforced.
It’s the kind of nuance that’s obvious to a human pentester, but easily missed when reviewers place a little too much confidence in AI-generated code.
AI Automation Complacency
There’s a well-documented idea from aviation that supervising automation takes more cognitive effort than performing the task manually. The same effect seemed to show up here.
Because the code wasn’t ours in the strict sense — we didn’t write the lines ourselves — the mental model of how it worked wasn’t as strong, and review suffered.
The comparison to aviation ends there, though. Autopilot systems have decades of safety engineering behind them, whereas AI-generated code does not. There isn’t yet an established safety margin to fall back on.
This Wasn’t an Isolated Case
This wasn’t the only case where AI confidently produced insecure results. We used the Gemini reasoning model to help generate custom IAM roles for AWS, which turned out to be vulnerable to privilege escalation. Even after we pointed out the issue, the model politely agreed and then produced another vulnerable role.
It took four rounds of iteration to arrive at a safe configuration. At no point did the model independently recognize the security problem – it required human steering the entire way.
Experienced engineers will usually catch these issues. But AI-assisted development tools are making it easier for people without security backgrounds to produce code, and recent research has already found thousands of vulnerabilities introduced by such platforms.
But as we’ve shown, even experienced developers and security professionals can overlook flaws when the code comes from an AI model that looks confident and behaves correctly at first glance. And for end-users, there’s no way to tell whether the software they rely on contains AI-generated code, which puts the responsibility firmly on the organizations shipping the code.
Takeaways for Teams Using AI
At a minimum, we don’t recommend letting non-developers or non-security staff rely on AI to write code.
And if your organization does allow experts to use these tools, it’s worth revisiting your code review process and CI/CD detection capabilities to make sure this new class of issues doesn’t slip through.
We expect AI-introduced vulnerabilities to become more common over time.
Few organizations will openly admit when an issue came from their use of AI, so the scale of the problem is probably larger than what’s reported. This won’t be the last example — and we doubt it’s an isolated one.
Book a demo to see how Intruder uncovers exposures before they become breaches.
Author
Sam Pizzey is a Security Engineer at Intruder. Previously a pentester a little too obsessed with reverse engineering, currently focused on ways to detect application vulnerabilities remotely at scale.
Sponsored and written by Intruder.

Artificial Intelligence
Cybersecurity
Honeypot
Intruder

Previous Article
Next Article

Comments have been disabled for this article.

Popular Stories

Fortinet admins report patched FortiGate firewalls getting hacked

Hackers breach Fortinet FortiGate devices, steal firewall configs

Zendesk ticket systems hijacked in massive global spam wave

Sponsor Posts

Identity Governance & Threat Detection in one: Get a guided tour of our platform

Discover how phishing kits are sold and deployed. Download the full research report.

Exposure Management Index: Insights From 3,000+ Teams. Get The Report.

Overdue a password health-check? Audit your Active Directory for free

Main Sections

News
Webinars
VPN Buyer Guides
SysAdmin Software Guides
Downloads
Virus Removal Guides
Tutorials
Startup Database
Uninstall Database
Glossary

Community

Forums
Forum Rules
Chat

Useful Resources

Welcome Guide
Sitemap

Company

About BleepingComputer
Contact Us
Send us a Tip!
Advertising
Write for BleepingComputer
Social & Feeds
Changelog

Username

Password

Remember Me

Not a member yet? Register Now

Reporter

Help us understand the problem. What is going on with this comment?

Spam

Abusive or Harmful

Inappropriate content

Strong language

Other

Read our posting guidelinese to learn what content is prohibited.

Submitting...
SUBMIT

Here’s a detailed summary of the provided text, focusing on the key insights and concerns raised about the use of AI in code generation and security reviews:

**AI-Generated Honeypots: A Cautionary Tale for Trusting Machines**

The article, sponsored by Intruder, presents a real-world case study highlighting the potential security risks associated with relying on AI-generated code, particularly within honeypot environments. It underscores the importance of maintaining a critical and discerning approach when evaluating code produced by AI tools, even when those tools appear confident and accurate.

**The Incident:** Intruder deployed an AI-assisted honeypot to collect early exploitation attempts. The AI, prompted to draft the honeypot infrastructure, inadvertently introduced a security vulnerability – the misinterpretation of client-supplied IP headers as the visitor’s IP address. This allowed an attacker to spoof their IP and inject payloads. The vulnerability was missed by both Semgrep OSS and Gosec, highlighting a crucial limitation of static analysis tools when faced with novel or nuanced issues.

**AI’s Role and the Human Factor:** The incident stemmed from a failure of human oversight. The lack of deep contextual understanding – a key element of a seasoned penetration tester’s approach – allowed reviewers to place undue trust in the AI’s output. The article effectively illustrates how AI-assisted development can lead to “AI automation complacency,” mirroring research on autopilot systems where reduced cognitive effort can diminish vigilance. Furthermore, the AI’s confident presentation of its solution, despite being untrained in security considerations, amplified this effect.

**Expanding Concerns Beyond the Honeypot:** This wasn’t an isolated event. The article details another instance where AI generated insecure IAM roles for AWS, requiring four iterations of refinement before a safe configuration was achieved. This demonstrates that AI models, even when capable of generating complex code, require substantial human guidance, particularly when dealing with security-sensitive tasks. The article suggests that organizations are likely underreporting the scale of this issue, as AI-introduced vulnerabilities are becoming more prevalent and organizations may be reluctant to admit their use.

**Implications for Development Practices:** The key takeaways for teams experimenting with AI-assisted coding are twofold. Firstly, the author recommends against allowing non-developers or non-security staff to rely on AI to write code, particularly in sensitive domains. Secondly, if experts *do* utilize these tools, a reassessment of the code review process and CI/CD (Continuous Integration/Continuous Deployment) detection capabilities is essential. The article anticipates a growing trend of AI-introduced vulnerabilities as organizations increasingly adopt these tools.

**Limitations of Current Technology:** The piece emphasizes that current AI models lack the contextual understanding and ‘security intuition’ developed through years of experience in penetration testing and secure coding practices. The models are demonstrably reliant on human steering, and their ability to independently recognize and address security problems is still nascent.

**Call to Action & Future Outlook:** The article concludes with a strategic recommendation: Organizations should book a demo with Intruder to understand how the company uncovers exposures before they become breaches. The author believes this issue will only become more important as AI tools evolve and their adoption increases. By proactively addressing the potential weaknesses of AI-generated code, organizations can reduce their exposure to a growing threat landscape.

---

Word Count: 968