AI and Liability: What Product Builders Need to Know Before Shipping

TL;DR

• Liability for AI outputs is shifting from users to builders: Courts and regulators are increasingly looking at who deployed the system, not just who pressed the button, especially when AI operates autonomously or users lack meaningful control.
• The "reasonable person" standard is being tested: Traditional negligence frameworks struggle with AI systems that can produce unpredictable outputs, creating uncertainty about what "reasonable care" means when deploying generative models.
• Product builders need proactive risk frameworks now: Waiting for settled law is too late—implement output monitoring, clear capability boundaries, and transparent limitations before regulatory enforcement catches up.
• Insurance and indemnification clauses won't save you: Standard tech liability protections weren't designed for generative AI's unique risks, and many policies explicitly exclude AI-generated content claims.

---

We're at an inflection point where AI liability is moving from conference room thought experiments to courtroom reality. As someone who's spent the last few years building AI products, I've watched this transition happen faster than most legal frameworks can adapt. The question is no longer if product builders will face liability for AI outputs—it's when and under what circumstances.

The recent discourse around AI accountability, particularly Simon Willison's analysis of liability frameworks, crystallizes something many builders are feeling but few are addressing systematically: we're operating in a legal grey zone that's rapidly darkening. And unlike previous waves of tech regulation that took years to materialize, AI liability is evolving in real-time, shaped by each new incident and lawsuit.

The Traditional Liability Framework Is Breaking

For decades, software liability followed a relatively predictable pattern. If your calculator app gave someone the wrong answer and they made a bad financial decision, you weren't liable—the user made the choice to rely on it. Software was treated as a tool, and tools don't typically create liability for their makers when used as intended.

AI fundamentally disrupts this model.

When a generative AI system produces content—whether it's legal advice, medical recommendations, or code—the line between "tool" and "actor" blurs. The system isn't just calculating based on explicit inputs; it's generating novel outputs that the builder couldn't have specifically foreseen. This autonomy changes the liability calculus.

Willison's piece highlights this tension effectively, noting that as AI systems become more capable and autonomous, the "tool defense" becomes harder to maintain. If your AI agent can independently make decisions, execute actions, and generate content without meaningful human oversight at each step, courts may start viewing it less like a calculator and more like an employee or contractor—entities for which you do bear liability.

Where Liability Actually Lands: Four Emerging Frameworks

1. Deployment Liability: You Shipped It, You Own It

The most straightforward framework emerging is deployment liability. If you put an AI system into production, you're responsible for its outputs within the scope of its intended use. This mirrors product liability law: if you manufacture and sell a defective product, you're liable for damages it causes.

For product builders, this means:

• Capability boundaries matter legally, not just technically: If your AI can generate medical advice but you market it as a "wellness companion," that distinction may not hold up in court if someone relies on it for health decisions.
• Guardrails are evidence of duty of care: Implementing content filters, output validation, and human-in-the-loop systems demonstrates you understood the risks and took reasonable steps to mitigate them.
• Scope of deployment defines scope of liability: An internal tool has different risk profiles than a consumer-facing product, and your liability exposure scales accordingly.

2. Negligence in Training and Testing

Even if you didn't directly deploy a model, if you trained it or fine-tuned it, you may face liability for negligent preparation. This framework asks: did you exercise reasonable care in ensuring the model was safe for its intended purpose?

The challenge here is that "reasonable care" for AI systems isn't well-defined. Unlike established engineering disciplines with codes of practice, AI development is still figuring out what constitutes adequate testing. However, some practices are emerging as baseline expectations:

• Red-teaming for harmful outputs
• Bias testing across demographic groups
• Adversarial testing for prompt injection and jailbreaks
• Documentation of known limitations

If you skip these steps and your model causes harm, a plaintiff's attorney will argue you were negligent in your duty of care.

3. Strict Liability for High-Risk Applications

Some jurisdictions and legal scholars are advocating for strict liability in high-risk AI applications—meaning you're liable for damages regardless of whether you were negligent. This mirrors how we treat inherently dangerous activities like blasting or keeping wild animals.

The EU's AI Act already categorizes certain applications as "high-risk" (healthcare, critical infrastructure, law enforcement), and we're likely to see strict liability frameworks emerge for these categories. For builders, this means:

• Risk classification is a product decision, not just a compliance checkbox: If your product touches high-risk domains, your entire development and deployment process needs to account for strict liability exposure.
• Insurance becomes mandatory, not optional: You cannot self-insure against strict liability in high-risk domains—you need actual coverage, which is still immature for AI products.

4. Vicarious Liability for AI Agents

The most legally novel framework treats AI agents as quasi-employees or contractors, making you vicariously liable for their actions within the scope of their "employment." This is where Willison's analysis becomes particularly relevant—as AI agents gain autonomy to act on behalf of users or organizations, traditional vicarious liability doctrines may apply.

If your AI agent sends emails, makes purchases, or provides advice "on behalf of" a user or company, and those actions cause harm, who's liable? The user who deployed the agent? The builder who created it? Both?

My take: this is where product builders have the most control and the most exposure. If you're building agentic AI—systems that act rather than just respond—you need to design for accountability from day one. This means:

• Audit trails for every action: Who initiated it, what context was used, what decision logic was applied.
• Explicit user consent for consequential actions: Don't let your agent send that email or make that purchase without human confirmation.
• Clear capability boundaries: Your agent should refuse tasks outside its competency scope, with explicit explanations of why.

What This Means for Product Strategy

I think the biggest mistake product builders are making right now is treating liability as a legal problem to be solved later, rather than a product constraint to be designed around from the start. Legal teams can't retrofit accountability into a system that was designed to be a black box.

Here's what I'm building into products now:

1. Transparency by Default
Every AI output should come with some indication of confidence, limitations, and source reasoning. This isn't just good UX—it's evidence that you're not misleading users about your system's capabilities. When something goes wrong, you want to show you were clear about limitations.

2. Staged Deployment with Risk Gating
Don't launch your AI feature to 100% of users on day one. Start with internal use, then controlled pilots, then gradual rollout with monitoring at each stage. This demonstrates you took reasonable care in deployment and gives you evidence of due diligence.

3. Human-in-the-Loop for Consequential Decisions
If your AI is making decisions that materially affect people—hiring, lending, medical treatment, legal advice—there should be meaningful human review. "Meaningful" means the human has context, expertise, and actual authority to override the AI.

4. Output Monitoring and Kill Switches
You need real-time monitoring of what your AI is producing in production, with automatic alerts for concerning patterns and the ability to immediately disable features if needed. This is table stakes for demonstrating you maintained reasonable oversight.

5. Clear Terms of Service and Capability Disclaimers
Yes, TOS won't shield you from all liability, but they establish what you told users about your system's capabilities and limitations. Be specific about what your AI can and cannot do, and what users should and should not rely on it for.

The Insurance Gap Nobody's Talking About

Here's an uncomfortable reality: most standard tech E&O (errors and omissions) insurance policies explicitly exclude AI-generated content claims, or have sub-limits so low they're effectively useless. The insurance industry is still figuring out how to underwrite AI risk, and the policies that do exist are expensive and limited.

This means two things for builders:

First, you're likely more exposed than you think. That insurance policy your company has? Read the exclusions carefully. Many policies written before 2023 don't contemplate generative AI at all, and those written after often have broad AI exclusions.

Second, insurance becomes a product constraint. If you can't get adequate coverage for a feature, that's a signal about its risk profile. Either redesign to reduce risk, or accept that you're self-insuring—which means having reserves or being prepared for existential risk if something goes wrong.

Regulatory Enforcement Is Coming Faster Than You Think

The EU's AI Act is already in force with phased compliance deadlines. California's AI transparency bills are moving through the legislature. The FTC has made clear it will use existing consumer protection authority to go after deceptive or harmful AI practices.

But here's what worries me more than regulation: private litigation. It only takes one sympathetic plaintiff with a good attorney to establish precedent. And unlike regulatory enforcement, which tends to target the biggest players first, private litigation can hit any company at any scale.

The first wave of AI liability cases is already starting:

• Defamation suits over AI-generated false statements
• Copyright infringement claims over training data and outputs
• Discrimination claims over biased AI decision-making
• Consumer protection claims over misleading AI capabilities

Each of these cases is establishing precedent about what "reasonable care" means for AI systems, what duties builders owe to users and affected parties, and what damages courts will award.

Building for Accountability: A Framework

So what does it actually look like to build AI products with liability in mind? Here's the framework I use:

Phase 1: Risk Assessment Before You Build

• What's the worst-case output your AI could produce?
• Who would be harmed and how severely?
• What's your intended use vs. foreseeable misuse?
• What risk category does this fall into (low, medium, high, unacceptable)?

Phase 2: Design for Controllability

• Can users understand what the AI is doing and why?
• Can they meaningfully control or override it?
• Are there guardrails that prevent the worst-case outputs?
• Is there a clear escalation path when the AI is uncertain?

Phase 3: Test for Failure Modes

• Red-team for adversarial inputs and prompt injection
• Test edge cases and demographic bias
• Validate outputs against ground truth where possible
• Document known limitations and failure modes

Phase 4: Monitor in Production

• Real-time output monitoring with alerts
• User feedback loops for problematic outputs
• Regular audits of production behavior vs. test behavior
• Incident response plans for when things go wrong

Phase 5: Document Everything

• Design decisions and risk tradeoffs
• Testing methodology and results
• Deployment process and rollout plan
• Monitoring and response procedures

This documentation isn't just for compliance—it's your evidence that you acted reasonably when (not if) something goes wrong.

The Uncomfortable Truth

Here's my honest take: I think we're building AI products faster than we understand their liability implications, and the reckoning is going to be painful for a lot of companies. The "move fast and break things" ethos that worked for consumer social apps doesn't work when you're deploying systems that can generate false medical advice, biased hiring decisions, or defamatory content at scale.

The good news is that liability risk is manageable if you design for it from the start. The bad news is that retrofitting accountability into systems designed without it is expensive, slow, and often technically infeasible.

Willison's analysis is right that we're in a transitional period where legal frameworks are still forming. But that doesn't mean we should wait for clarity before acting. The companies that will succeed in the long term are those treating liability as a first-order product constraint, not a second-order legal problem.

What to Do Monday Morning

If you're building AI products, here's what you should do this week:

1. Audit your current AI features for liability exposure using the four frameworks above. Where do you fall on each?

2. Review your insurance coverage with your broker specifically asking about AI-generated content claims. Get it in writing.

3. Implement output monitoring if you haven't already. You cannot manage risk you're not measuring.

4. Document your risk assessment and mitigation strategy. Even a simple one-pager is better than nothing.

5. Add human review for your highest-risk outputs. Start with the top 10% riskiest use cases.

The window where AI liability is still uncertain is closing. The companies that prepare now will have a massive advantage over those that wait for the first lawsuit to think about accountability.

Liability isn't just a legal constraint—it's a product quality signal. If you can't deploy your AI feature in a way that manages liability risk, that's telling you something important about whether it's ready to ship. Listen to that signal.