AI Is Breaking Two Vulnerability Cultures: What Product Builders Must Know Now

Last month, a security researcher discovered a prompt injection vulnerability in a major AI assistant that could leak user data. They followed the traditional responsible disclosure process: documented the issue, contacted the vendor privately, waited 90 days, then published. The vendor's response? "This is expected behavior, not a vulnerability."

Welcome to the new reality of AI security, where two decades of carefully constructed vulnerability management practices are crumbling in real-time.

As someone who's shipped AI products to millions of users, I've watched this collision happen in slow motion. The frameworks that worked brilliantly for traditional software—CVE databases, coordinated disclosure timelines, clear severity ratings—are buckling under the weight of AI's fundamentally different risk profile. And if you're building AI products today, this isn't just a security team problem. It's your problem.

The Two Cultures That Shaped Modern Security

To understand what's breaking, we need to understand what we built.

Culture One: The Coordinated Disclosure Framework

For the past two decades, the security industry operated on a social contract. Researchers would find vulnerabilities, report them privately to vendors, give them time to patch, then disclose publicly. This created a predictable rhythm:

• Day 0: Researcher discovers vulnerability
• Day 1: Private notification to vendor
• Days 1-90: Vendor develops and tests patch
• Day 90: Coordinated public disclosure
• Day 90+: Users patch their systems

This system worked because vulnerabilities were discrete, fixable, and had clear boundaries. A buffer overflow is a buffer overflow. You patch it, ship the update, and the vulnerability is gone.

The incentives aligned beautifully. Researchers got credit and recognition. Vendors got time to fix issues before exploitation. Users got security updates. Everyone won.

Culture Two: The CVE Severity System

Parallel to disclosure practices, we built a sophisticated taxonomy for categorizing vulnerabilities. The Common Vulnerabilities and Exposures (CVE) system gave us a shared language. CVSS scores told us how serious things were:

• 9.0-10.0: Critical - Drop everything
• 7.0-8.9: High - Patch this week
• 4.0-6.9: Medium - Next sprint
• 0.1-3.9: Low - Backlog it

This worked because traditional software vulnerabilities had measurable, predictable properties. You could calculate exploitability based on attack complexity, required privileges, and user interaction. The math worked.

Security teams built entire operational frameworks around these scores. Compliance requirements referenced them. Insurance policies were written around them. Boards asked about them in quarterly reviews.

Why AI Breaks Both Cultures Simultaneously

AI doesn't just bend these frameworks—it shatters their foundational assumptions.

The Unpatchable Vulnerability Problem

Consider prompt injection, the most widely discussed AI vulnerability. Unlike a buffer overflow, you can't simply patch it away. It's not a bug in the code—it's an emergent property of how language models process text.

When a researcher reports a prompt injection technique, what exactly should the vendor do? There's no line of code to fix. You might add guardrails, but those can be bypassed with slightly modified prompts. You might retrain the model, but that takes months and doesn't guarantee immunity to new variations.

This breaks the coordinated disclosure timeline. What does "patched" even mean when the vulnerability is probabilistic rather than deterministic? When I've received these reports for products I've built, the honest answer is often: "We've reduced the likelihood of this attack vector, but we cannot eliminate it."

That's not what the disclosure framework was designed to handle.

The Severity Scoring Impossibility

How do you assign a CVSS score to a vulnerability that:

• Works 60% of the time, depending on model temperature and context?
• Might leak sensitive data, or might just generate nonsense?
• Can be triggered by users accidentally, but requires specific phrasing?
• Has impact that varies wildly based on the application's trust boundaries?

I've sat in severity assessment meetings where we spent hours debating whether a jailbreak technique was "High" or "Medium" severity, only to realize the entire framework didn't apply. The attack vector was "text input"—which is literally the primary interface. The privileges required were "none"—anyone can send a prompt. The impact was "variable"—sometimes concerning, sometimes harmless.

The math doesn't work when the vulnerability is stochastic.

The Disclosure Incentive Collapse

Here's where it gets worse: researchers are losing motivation to follow responsible disclosure.

When you report a traditional vulnerability and the vendor patches it, you get:

• Credit in the security advisory
• A CVE number with your name on it
• Sometimes a bug bounty payment
• Recognition from the security community

When you report an AI vulnerability, you often get:

• "This is expected behavior"
• "We're aware of this class of issues"
• No CVE, no credit, no bounty
• Your technique published by someone else three weeks later

The incentive structure has collapsed. Why follow a 90-day disclosure timeline when the vendor won't acknowledge it as a vulnerability, and your technique will be independently discovered and published by a dozen other researchers before your disclosure window closes?

What This Means for Product Builders

If you're building AI products, you're navigating a security landscape without established rules. Here's what I've learned from operating in this environment:

1. Redefine "Vulnerability" for Your Context

Stop waiting for the industry to settle on definitions. Define what constitutes a security issue for your specific product.

For an AI coding assistant, perhaps a vulnerability is: "A reproducible technique that causes the model to generate code with security flaws at >80% frequency." For a customer service chatbot: "A prompt pattern that reliably extracts PII from previous conversations."

Make these definitions public. Put them in your security policy. Give researchers clear targets.

2. Build Layered Defenses, Not Patches

Accept that you cannot "fix" most AI vulnerabilities. Instead, build defense in depth:

• Input filtering: Not to stop all attacks, but to raise the bar
• Output validation: Catch obvious failures before they reach users
• Rate limiting: Prevent automated exploitation at scale
• Monitoring: Detect unusual patterns that might indicate attacks
• Isolation: Limit blast radius when (not if) something goes wrong

When I design AI product architectures now, I assume the model will eventually do something unexpected or malicious. The question isn't "how do we prevent this?" but "how do we contain it?"

3. Create New Incentive Structures

The traditional bug bounty model doesn't work well for AI, but researchers still need incentives to engage responsibly.

Consider:

• Paying for novel attack classes, not individual instances
• Recognizing researchers who help improve defenses, even without traditional CVEs
• Creating public leaderboards for red-teaming contributions
• Offering early access to new models for security researchers

At one company I advised, we created a "AI Safety Researcher" recognition program that acknowledged contributions to understanding model behavior, not just finding exploits. Engagement increased 3x.

4. Communicate Probabilistic Risk Clearly

Your users, executives, and compliance teams are used to deterministic security statements: "This vulnerability is patched." "This attack is prevented."

You need to educate stakeholders on probabilistic security:

• "This attack succeeds in approximately 15% of attempts, down from 45% before our mitigations."
• "We detect and block 94% of prompt injection attempts in production."
• "The expected value of data leakage from this attack vector is $X per year."

This is uncomfortable. Executives want certainty. Compliance frameworks demand it. But false certainty is worse than uncomfortable truth.

5. Contribute to Emerging Standards

The vulnerability management framework for AI is being written right now, in real-time, through the collective actions of builders, researchers, and users.

Participate:

• Share your security incidents and learnings publicly
• Contribute to efforts like OWASP's LLM Top 10
• Engage with researchers, even when their findings are uncomfortable
• Document what works and what doesn't

The companies that help shape the new framework will have a competitive advantage when it solidifies.

The Broader Implications

This isn't just about vulnerability management—it's a preview of how AI breaks many established frameworks.

We're seeing similar pattern-breaking in:

• Compliance: How do you audit a system that behaves differently every time?
• Testing: What does "test coverage" mean for emergent behavior?
• Liability: Who's responsible when an AI causes harm probabilistically?
• Quality assurance: How do you define "acceptable" for stochastic outputs?

Every framework built on deterministic assumptions is facing a reckoning.

Navigating the Transition

The vulnerability cultures we built over two decades served us well. They made software meaningfully more secure. But they were designed for a different paradigm.

AI isn't just incrementally different—it's categorically different. Vulnerabilities aren't bugs to be fixed; they're risks to be managed. Security isn't a state to achieve; it's a continuous process of measurement and mitigation.

For product builders, this transition period is treacherous but also opportunistic. The companies that figure out how to build and operate secure AI systems under the new paradigm will have a significant competitive advantage. Those that try to force AI into the old frameworks will face escalating security incidents and researcher backlash.

Here's what I'm doing in my own work:

Accepting uncertainty: Building systems that degrade gracefully when models behave unexpectedly, rather than assuming perfect behavior.

Measuring continuously: Instrumenting everything to understand actual attack patterns, not theoretical vulnerabilities.

Engaging openly: Treating security researchers as partners in understanding model behavior, not adversaries to be managed.

Educating stakeholders: Helping executives, users, and regulators understand the new risk landscape, even when it's uncomfortable.

Iterating rapidly: Treating security as a continuous improvement process, not a release checklist item.

The old vulnerability cultures are breaking. That's not a crisis—it's an opportunity to build something better, something designed for the reality of AI systems rather than the convenience of traditional software.

The question isn't whether the old frameworks will break. They already have. The question is what we'll build to replace them, and whether we'll do it thoughtfully or reactively.

As builders, we get to choose. Choose wisely.