Who is liable when an AI agent gives a wrong answer?
Short answer: In two May 2026 German rulings, the company that deployed the AI was held responsible for what the AI said. Neither ruling is final, and this is not legal advice. For engineering teams, the practical answer is that your company can only defend an agent's output if you can show who owned the agent, what it did at each step, and which risky decisions a human approved.
Your agent will tell a customer something false. Nobody on your team will have written that sentence. The model will have.
I discussed this on Diary of a CTO with Chen Salomon, VP Engineering at Sparta Commodities, who spent more than six years at monday.com leading the AI governance work that lets enterprise customers control how agents run.
What did the German courts decide in May 2026?
Hamm, 12 May 2026. The Higher Regional Court of Hamm (case 4 UKl 3/25) ruled against a clinic operator after its website chatbot told visitors that two managing directors held specialist titles that do not exist. The clinic said it had fed the bot only correct data. The court held it responsible anyway and treated the chatbot as the company's own conduct, not a third party. The court allowed an appeal to the Federal Court of Justice.
Munich, 28 May 2026. The Munich Regional Court I (case 26 O 869/26) issued a preliminary injunction against Google. Its AI Overviews had linked two Munich publishers to scams and subscription traps. The court treated the overviews as Google's own statements.
These are German decisions under German law, and neither is final. They still test two defenses many agent launches assume: "we cannot fully control the model" and "the user could have checked."
Why does "we only trained it on accurate data" fail?
This section is my reading, not Chen's. The clinic's defense described the inputs to the bot. The court judged the output. Defending inputs tells a court nothing about what the bot actually said. A team that can show the output and the steps behind it can answer the question the court asked.
Who should own an AI agent in production?
Chen's answer is that the builder who designed and shipped the agent owns it, the same way a developer owns a pull request that a coding agent wrote. Ownership includes design choices. He said an owner may decide not to verify an end state because verification costs latency or money. That is allowed, provided a person made the call and can explain it later.
A minimal ownership record (my suggestion): agent name, owner, backup owner, what it is permitted to say or do, which verification steps were deliberately skipped and why, and the date of the last model change.
What should an AI agent trace record?
Chen does not evaluate agents like a unit test, where the output is compared to an expected output. An agent runs as a graph of decisions. It picks a tool, reads the result and decides again. His example was a stock recommendation. If the fundamentals search returns an earnings figure five times too high, the "buy" can be correct and still rest on a false fact. So he asks which facts the agent had and which citations it used at each step.
For a refund agent, his check is concrete: did it ask for the product details, fetch the invoice and authenticate that the requester made the purchase?
A minimal trace record, one per step (my suggestion):
{
"run_id": "r_8841",
"step": 4,
"agent": "refund-agent",
"owner": "j.doe",
"model_version": "provider-model-2026-09",
"tool_called": "fetch_invoice",
"sources_cited": ["invoice_20991", "policy_v7"],
"output_summary": "Purchase confirmed, within 30-day window",
"route": "auto | human_online | human_offline",
"reviewer": null,
"cost_tokens": 2140,
"timestamp": "2026-10-20T09:14:02Z"
}
If a run is stored only as input and output, you can test the answer and nothing else. The step-level record is what lets you find a false fact buried under a correct conclusion.
How should risky agent decisions be routed to a human?
Engineers built permissions as yes or no. Chen adds a third value, "maybe": the agent goes so far, then a human decides. His refund example has the agent read the receipt, confirm the purchase, take the reason and hand a summary to a person. He described two flavors. Online, the agent waits for a human, which costs time and freshness. Offline, a reviewer marks decisions after the fact and you watch the results on a dashboard. His condition for both is that the decision is documented so it can be judged later. He said this is hard to maintain and a big mindset shift.
A starting routing table (my suggestion, tune it to your risk):
| Risk tier | Example | Route |
|---|---|---|
| Low | Answering from approved help-center pages | Auto, with sampled offline review |
| Medium | Recommendations built on retrieved facts | Auto above a confidence threshold, otherwise human |
| High | Refunds, legal or medical statements, claims about a person's credentials | Human before the response is sent |
The Hamm case sits in the high row: a statement about someone's professional title is the kind of claim that should not ship unreviewed.
How do you set a budget for an agent that decides its own cost?
Chen said cost predictability is the first question buyers ask, ahead of quality and privacy, and that pricing for software that makes a variable number of model calls is hard. He expects budgets to follow business priority: a critical process gets more headroom than an internal marketing tool. Every extra verification step costs money and latency, so each one is a spend decision.
I asked what happens to customer data when an agent hits its cap at step 40 of 300. Chen did not give a mechanism, and I don't think a standard one exists yet. Decide before launch whether half-finished work rolls back or completes, and write the answer into the agent's ownership record.
How should you test a new model version before upgrading?
Chen's position is that a lab saying a model is better is not evidence about your product, because behavior and parameters change behind the scenes. He treats an upgrade like a feature and runs an A/B test, whether the move is to a stronger model or a cheaper one.
A protocol that follows from this (my suggestion):
- Replay a fixed set of real production traces on the old and new model.
- Compare at the step level: tool choice, sources cited, factual accuracy of each claim, cost and latency.
- Write the rollback criteria before the test starts.
- Roll out to a slice of traffic first, and update the ownership record with the new model version.
Checklist before the next agent ships
- One named owner per agent, with a backup.
- A step-level trace for every run.
- High-risk decisions routed to a human, with every approval or rejection recorded.
- A written decision on what happens when the cost cap is hit mid-run.
- An A/B test before every model change.
- Deterministic controls where possible, such as data residency, human review and logging, and an honest "we don't know" about the rest.
FAQ
Can a disclaimer like "this is AI, check the answer" remove liability? In Munich, Google's position that users can check sources did not stop the preliminary injunction. Chen's view is that generic warnings put too much on users who cannot tell what a correct answer looks like. He prefers specific context, such as showing which page an answer came from, so the user can make an informed decision.
Does this apply outside Germany? Both rulings are German, and neither is final. Chen said he is not a legal expert and that the vendor-versus-customer liability line is still an open question for enterprises, especially when the customer builds the agent on a platform. Treat the rulings as a signal and take legal advice for your jurisdiction.
How often can an agent be wrong before you cannot ship it? I asked Chen, and neither of us had a number. If your team has one and a way to measure it, reply and tell me.
Himanshu Saxena hosts Diary of a CTO and writes about AI governance and engineering leadership.