The VibeSec Reckoning

Refactoring
Agile
Architecture
About
Thoughtworks

Topics
Architecture
Refactoring
Agile
Delivery
Microservices
Data
Testing
DSL

about me
About
Books
FAQ

content
Videos
Content Index
Fragments
Board Games
Photography

Thoughtworks
Home
Insights
Careers
Radar
Engineering

follow
RSS
Mastodon
LinkedIn
Bluesky
X
BGG

Table of Contents

Top
What we learned the hard way
The numbers behind the risk
The real problem: prompts are not enough
Why business functions need to pay attention
Short-term habits
Medium-term solutions
The security context file in practice
The security intelligence feed in practice
Long-term organisational changes
Conclusion: Scaling Beyond the Prototype

The VibeSec Reckoning
Why prompting your AI to “be secure” is not enough, and what
actually is

“Vibe coding” - the practice of non-technical citizen builders using
generative AI tools to rapidly develop applications, this has significantly accelerated
software prototyping. However, because AI agents naturally prioritise the path of
least resistance, they frequently recommend insecure configurations, creating systemic
security exposure across industries. To combat this we need to write a
security context file to guide the AI, be cautious with AI permission
requests, create a daily security intelligence feed, and provide builders with
a secure-by-default harness and templates.
27 May 2026

Gautam Koul

Gautam is Head of AI applications, Global Marketing at Thoughtworks.
He leads AI platform initiatives and applied AI engineering teams focused on building and scaling production-ready,
token-efficient GenAI applications across the Google ecosystem.

Lucian Moss

Lucian is an AI Engineer in Global Marketing at Thoughtworks,
specialising in Retrieval Augmented Generation and agentic systems.

Neil Drew-Lopez

Neil is an AI Engineer in Global Marketing at Thoughtworks, specialising in data engineering,
multi-agent systems, and research into new AI capabilities.

Daberechi Ruth Edeokoh

Ruth is an AI engineer in Global Marketing at Thoughtworks who builds intelligent systems and AI agents that turn complex data into practical insights and scalable,
real-world solutions.

security
generative AI

Contents

What we learned the hard way
The numbers behind the risk
The real problem: prompts are not enough
Why business functions need to pay attention
Short-term habits
Medium-term solutions
The security context file in practice
The security intelligence feed in practice
Long-term organisational changes
Conclusion: Scaling Beyond the Prototype

Vibe coding is enabling non-technical users (or as we call them, citizen
builders) to build applications with AI that they simply could not have
built before. When our AI applications team in Global Marketing at
Thoughtworks was asked to scale a vibe coded prototype built by one of our
citizen builders in global marketing, we discovered serious cracks that
prevent vibe coded applications from going into production safely.
Speed without guardrails is a risk no team can afford to ignore. What
follows is the story of what we found, what it means for teams building with
AI, and the steps we are taking to make sure every workflow, prototype, and
app we ship is one we can stand behind.

What we learned the hard way
The AI applications team within Global Marketing was asked to scale a video
assembly prototype built with Gemini, Replit AI and Claude AI to create
on-brand videos to be used across our 10,000 employees. The team ran into two
moments that stopped work cold. In both cases, the AI suggested a path with
serious security implications. In both cases, it took a human asking the right
question to catch it.

Security risk # 1
Public storage access
The AI recommended making the storage
bucket public, or setting cloud file storage to “anyone with the link.” When
challenged, it justified this by saying every company does it. Only a firm
rejection prompted a secure alternative.
This could have leaked sensitive
unreleased brand assets and audience data to the public internet.

Security risk # 2
Excessive token permissions
A service account was assigned the Access
Token Creator role, granting it the ability to create short-lived tokens and
access databases and other resources far beyond what the task required. The team
caught this before running the code.
This would have allowed a compromised
service account to move laterally through an entire cloud workspace.

The key insight here is that AI tools often suggest the path of least
resistance. That path is not always the secure one. Human judgment remains
essential, but it should not be the only control. The goal is to give
agents technical security rules as context from the first prompt, then
validate their output through deterministic checks in the development
workflow so insecure code, permissions, secrets, or infrastructure cannot
pass unnoticed.

The numbers behind the risk

44%
Rise in attacks exploiting application vulnerabilities, year on year

1 in 5
Enterprise breaches now caused by AI-generated code

50%
Organisations with no sensitive data policies for AI

25%
AI-generated code with confirmed vulnerabilities

These incidents are not isolated. Research published in 2026 confirms that
AI-assisted coding at speed creates systemic security exposure. The same risks
we encountered are playing out across the industry right now.

FindingStatSource

AI-generated code with confirmed vulnerabilities25%AppSec Santa, 2026
Rise in attacks exploiting application vulnerabilities, year on year44%SQ Magazine AI Coding Security Statistics, 2026
Codebases with high or critical severity vulnerabilities78%Black Duck OSSRA 2026
Organisations with no sensitive data policies for AI50%AppSec Santa, 2026
Mean vulnerabilities per codebase increase year on year+107%Black Duck OSSRA 2026
Enterprise breaches now caused by AI-generated code1 in 5Aikido Security, 2026
New CVEs from AI-generated code in March 2026 alone35Georgia Tech Vibe Security Radar, SSLab, March 2026
AI systems with prompt injection exposure in 2026 audits73%SQ Magazine AI Coding Security Statistics, 2026
Share of all new enterprise software that is AI-generated42%Sonar developer survey, 2026
Security teams say keeping up with AI-generated code volume is getting harder62%ProjectDiscovery AI Coding Impact Report, April 2026

The real problem: prompts are not enough
After sharing these incidents with engineering and security colleagues, a
clear message came back: telling an AI agent to be safe is not the same as
enforcing that it is safe. Prompts can be overridden, misunderstood, or
ignored. The moment a user pushes back on a restriction, or phrases a request
differently, the constraint can evaporate.

“It is not sufficient to merely tell the LLM the desired behavior of your output artifacts. If you absolutely do not want something to be true, it must be codified in non-negotiable rules somewhere in your development lifecycle.” - Engineering leadership

Think of it this way: prompting for test-driven development is not the
same as enforcing code coverage thresholds in your build tool. One is a
suggestion. The other is a gate. Birgitta
Böckeler’s work on harness engineering makes this concrete by
outlining a mental model for building trust in coding agents. Instead of
relying solely on prompts, developers wrap the agent in an outer “harness”
structured along two axes:

Guides (feedforward controls) anticipate unwanted
behavior and steer the model before it acts, while Sensors (feedback controls)
observe the code after the agent acts to flag errors.
Computational controls are deterministic,
fast, and CPU-run (like linters or test suites). Inferential controls rely on
semantic analysis and AI-driven judgment (like specific system prompt
constraints).

Why business functions need to pay attention
Business functions like our marketing team, who are building with AI, are
not exempt from the security obligations that apply to engineers building
applications. Building security into software is a fundamental requirement for
protecting customer and employee data. Even lightweight internal prototypes
must comply with enterprise security standards. Without the right guardrails,
AI-assisted development can expose sensitive data long before an application
reaches production.

Client trust
Compliance is contractual
Adhering to standards like ISO 27001 ensures the protection of
sensitive data. All applications, regardless of how quickly they are
built, must meet these security benchmarks to maintain the trust of
customers and employees.

Brand integrity
Brand assets need protection
Core work involves sensitive functional data (e.g., unreleased
campaign assets, financial data, or audience insights).
Over-permissioned service accounts put far more than code at
risk.

Reputation
Business functions can set the standard
When business functions like marketing lead with security
discipline, they signal responsible AI adoption to the wider
organisation and to clients.

Short-term habits
You do not need to be a security expert to start building responsibly.
These three habits can get you started.

Feed your technical security rules into every
session
Add your organisation's security guidelines as “Rules” in Claude,
Cursor, or Replit to begin with (later on invest in a shared sensible
default layer across all tools). The AI agent uses them as guidance,
making secure patterns more likely from the start. They still need to
be backed by deterministic checks that fail unsafe code, exposed
secrets, broad permissions, vulnerable dependencies or insecure
infrastructure before anything is deployed.

Question every permission the AI suggests
If a tool recommends making something public or assigning a
broad service account role, stop and ask why. The path of least
resistance and the secure path are rarely the same thing.

Try the red team prompt
Ask your AI to roleplay as a bad actor and pen test what it
just built. This technique consistently surfaces vulnerabilities that
forward-looking prompts miss, particularly around permissions and data
exposure.

Medium-term solutions
Reading about risk is one thing. Doing something about it is another. These
incidents prompted two practical initiatives. The principles behind them are
replicable by any team building with AI, regardless of technical
background.

A security context file
We compiled our technical security rules into a structured context
file loaded into every AI coding session before any code is written.
It covers zero trust enforcement, secrets management, harness
engineering and supply chain integrity. The key distinction from a
casual prompt is operational discipline: the file is versioned, loaded
by default, reviewed, and paired with automated checks. It acts as a
comprehensive inferential guide telling the agent what good looks
like; but it must be paired with computational sensors in the pipeline
to validate whether the output is acceptable.

A daily security intelligence feed
Currently, this automated consolidation ensures we see supply chain
alerts the day they are published. In the future, as we work toward an
agentic enterprise, we envision agents proactively creating story
cards and identifying and patching known vulnerabilities behind the
scenes for human review, significantly reducing the Software
Development Lifecycle (SDLC) cycle time.

The security context file in practice
The idea behind this approach is straightforward: AI tools read context at
the start of a session, so make that context your technical security rules.
The file is the result of working through your organisation's security
requirements and structuring them in a form the AI can act on, not just
acknowledge.
What follows is the kind of coverage any such file should include. The
specifics will differ by organisation, but the categories are consistent.

Area coveredWhat good looks likeWhy it matters

Zero trust and least privilegeStrict identity verification and minimum access rights on every service account and storage resourceSets the inferential guide parameters to prevent the token permission risk directly.
Secrets managementAI refuses to generate or store API keys, passwords or tokens in code; always routes to environment variables or a secrets managerStops credential leakage before it reaches a repository.
Harness engineering gatesSAST scanning, credential scanning and infrastructure validation must pass before deployment; no reliance on prompt instructions aloneBacks up inferential instructions with deterministic, computational sensors.
Supply chain integrityOnly well-established libraries; regular audits of every dependency for known vulnerabilitiesReduces risk from AI suggesting obscure or unvetted packages.
AI accountabilityAll AI-generated code is flagged for peer review and automated security scanning before deployment; no unsanctioned AI usageRequired for compliance auditability.

The key distinction from a prompt is that the file contains non-negotiable rules that force the AI agent to refuse requests that violate policy. If the AI agent is asked to bypass a check, disable logging or set something to public access, the rules should steer it to decline and explain why. But the important control is that deterministic checks and deployment gates should still catch the issue even if the agent fails to follow that guidance. That refusal is precisely what was missing in both of our near-miss incidents.

The security intelligence feed in practice
Staying informed is its own form of defence. The workflow monitors the
tools and languages your team actively uses and delivers a daily digest of new
CVEs, platform advisories and security bulletins. The coverage areas that
matter most are consistent across organisations: the languages they write in,
the cloud platforms they deploy to, the AI coding tools themselves, and the
CVE database as a whole.
The goal is simple: learn about a vulnerability on the day it is disclosed,
not weeks later. At 42% of all new enterprise software now AI-generated or
AI-assisted, the tools that accelerate development are also the tools most
likely to appear in new CVE disclosures. Monitoring them actively is part of
owning your security.

The Broader point
Neither of these approaches requires an engineering background to
adopt. One is a policy document structured for AI consumption. The other
is an automated search. What they share is the recognition that passive
security awareness is not enough when AI is generating code at
speed.

Long-term organisational changes

From prompts to pipelines
Integrate harness engineering into your standard prototyping
templates. Move from probabilistic prompts to explicit feedback loops.
If a computational sensor (like an automated security scanner)
triggers, the agentic loop must structurally force the model to
self-correct until it passes.

Feed security rules into the application builder (Cursor, Claude etc)
Compile your organisation's technical security rules into a
structured context markdown file and load it as “Rules” which the
model will have to adhere to. It catches the most common missteps at
the point where they are cheapest to fix, before any code is
committed.

Make the secure path the easy path
Give builders a secure-by-default starting point. Templates that
pre-configure authentication patterns, private storage defaults,
secrets handling and dependency scanning reduce the chance that anyone
takes a shortcut under deadline pressure.

Define a starter harness across functions
A shared starter harness built together by business functions,
engineering and security gives every builder a secure foundation from
day one, rather than each team independently rediscovering the same
mistakes.

Conclusion: Scaling Beyond the Prototype
This journey started when we were brought in to support another team
building a video assembly platform for a Global Marketing hackathon. As we
helped scale the solution, it became clear that “vibe coding” without
enterprise-grade guardrails can introduce risks that organizations simply
cannot overlook.
By embedding our technical security rules directly into the agent workflow,
we transformed those early near-misses into a secure, production-ready
platform that was successfully rolled out to 150 users during the
hackathon.
That shift - from depending on humans to catch issues, to building
technical security rules, automated checks and human accountability into the
workflow has become our blueprint for moving fast while maintaining
engineering rigor in the agentic era.

Further reading from Thoughtworks
These challenges are not unique to our team. Thoughtworks colleagues have
been working through them in depth and publishing their findings. These are
the pieces most directly relevant to what we described.
Thoughtworks CTO Rachel Laycock on why engineering rigour does not
disappear when AI generates code, it relocates. Covers putting coding agents
on a leash, zero trust architectures, and why security guardrails are now
non-negotiable.
A Thoughtworks-hosted gathering of
practitioners and enterprise leaders found that security
consistently gets deprioritised in AI adoption. Granting agents broad tool
access, particularly to email, was flagged as an immediate and specific
risk.
Birgitta Boeckeler, Distinguished Engineer at Thoughtworks, on placing
deterministic controls inside the agentic loop. The definitive guide to
building a harness that enforces security rules without paralysing the
model.
How to embed security guardrails and
never-allow rules as mandatory pre-commit hooks in any AI-assisted
development workflow.
Thoughtworks annual technology outlook warns that without rigorous
engineering oversight, generative tools risk compounding technical debt and
introducing security vulnerabilities. Success requires a co-construction
model where humans ensure AI-generated code maintains architectural
integrity.

Acknowledgments
Big thanks to the team for the engaging discussions that shaped this article, in particular Julie Woods-Moss for championing AI prototyping risks,
and Natalie Drucker for her significant role in shaping the content and title.
Thanks to Martin Fowler for reshaping our perspective on short-term versus long-term habits and harness engineering,
and to Kief Morris for anchoring the core argument on LLMs bypassing prompts.
Appreciations to Jim Gumbley for bringing in the critical governance and ISO27001 focus and reviewing the article.
Honorable mentions - Craig Stanley for advocating the strategic rollout, and Andherson Maeda for the concept of a centralized skills repository,
Simone Thompson for the bad-actor simulation methodology, Premanand Chandrasekaran for defining the technical boundaries of LLM enforcement,
Matteo Vaccari for the nuances on balancing strict rules with model flexibility, and Bilal Jaffery.
GenAI (Gemini and Claude) was used to streamline the research process, pull in insights, and polish the language for maximum clarity and readability.

Significant Revisions
27 May 2026: published

Topics
Architecture
Refactoring
Agile
Delivery
Microservices
Data
Testing
DSL

about me
About
Books
FAQ

content
Videos
Content Index
Fragments
Board Games
Photography

Thoughtworks
Home
Insights
Careers
Radar
Engineering

follow
RSS
Mastodon
LinkedIn
Bluesky
X
BGG

© Martin Fowler | Disclosures

The practice of "vibe coding," which involves non-technical builders using generative AI tools to rapidly prototype applications, has significantly accelerated development but introduced systemic security exposure because AI agents naturally favor the path of least resistance, often suggesting insecure configurations. This dynamic necessitates a shift in approach, moving beyond simply prompting AI to "be secure" toward enforcing security through structured, verifiable guardrails.

The discovery of this risk was highlighted when an AI applications team at Thoughtworks attempted to scale a prototype, encountering critical security issues when the AI suggested actions such as making storage buckets public or assigning overly permissive service account roles. These incidents revealed that relying solely on human judgment is insufficient; the solution requires embedding technical security rules within the development lifecycle. The core problem is that prompts alone are not enough, as constraints can be easily overridden or misunderstood. As noted by Birgitta Böckeler, achieving trust in coding agents requires moving beyond suggestive prompts to enforcing non-negotiable rules within the development process.

To manage this risk, the authors propose a multi-layered strategy. Short-term habits involve feeding organizational security guidelines directly into the AI sessions, critically questioning any security-related suggestions, and employing techniques like the red team prompt to surface potential vulnerabilities. Medium-term solutions focus on operationalizing these principles through concrete mechanisms. One key proposal is the development of a security context file, which compiles organizational security requirements—covering zero trust enforcement, secrets management, harness engineering, and supply chain integrity—into a structured context file loaded by default into every AI coding session. This context file acts as an inferential guide, but it must be paired with deterministic, computational sensors, such as static analysis and infrastructure validation, which serve as feedback controls to validate the agent's output before deployment.

Furthermore, maintaining awareness is essential through a daily security intelligence feed, which automatically consolidates alerts regarding new Common Vulnerabilities and Exposures (CVEs) and platform advisories relevant to the tools and environments used by the team. This approach addresses the fact that the tools accelerating development are also the sources of new vulnerabilities, demanding continuous monitoring.

Organizationally, long-term changes require integrating these concepts into standard workflows. This includes moving from probabilistic prompting to explicit feedback loops where computational sensors structurally force the agent to self-correct. Builders should be provided with secure-by-default starting points through templates that pre-configure secure patterns for authentication and storage. Establishing a shared starter harness, developed collaboratively by business, engineering, and security teams, ensures that every builder starts with a secure foundation. This shift transforms the process from waiting for humans to catch errors into building technical security rules and automated checks directly into the agentic workflow, ensuring that speed is achieved while maintaining engineering rigor in the age of AI development.