What is AI observability when an agent can fail without an error?
An agent can return a convincing answer, report that it completed a task and still be wrong. A successful request tells you that software responded. It does not establish that an autonomous system used correct information or stayed within its authority.
In this Governance File excerpt, Gaurav Maheshwari, co-founder of Oodle AI, discusses silent failures, agent traces and production permissions. The central operational question is how teams identify the important failures among large numbers of apparently successful runs.
Collecting traces is the beginning
A trace can record which model and tools an agent used and how the run progressed. That record is valuable, but collection alone does not decide whether the result was correct or the behaviour acceptable.
My practical interpretation is to examine three layers together:
| Layer | Question |
|---|---|
| Application health | Did the request complete, and at what latency and cost? |
| Task quality | Was the answer supported, and was the intended task completed? |
| Authority | Were the actions permitted for this agent and this request? |
A green dashboard at the first layer cannot answer the other two. Conversely, finding a bad answer does not tell a team which step introduced it. Teams need a way to connect the outcome to the recorded decisions.
A model upgrade can change what an agent tries
The published description of this clip includes Gaurav's account of a more capable coding model creating debug pods and inspecting binaries in a production Kubernetes cluster. The related permissions analysis treats this as his reported experience, not an independently verified incident.
The engineering implication is to review behaviour and permissions together. The same tool access can support a wider set of actions when the model becomes more capable. A new model should therefore prompt questions about where the agent can act and what would reveal an unexpected action.
Give the agent its own identity
If an agent inherits an engineer's broad access, it can reach systems unrelated to its immediate task. A separate identity makes it possible to restrict permissions and distinguish the agent's activity from the person's.
That identity also improves investigation. A trace should be linkable to the account that acted, the resource it reached, the tool result and any approval associated with the action.
Turn monitoring into a reviewable decision
My suggested starting point is to define a small set of questions an operator can actually answer:
- Did the agent use information supported by the available evidence?
- Did it touch a resource outside the approved task?
- Did a proposed production change receive the required approval?
- Did the tool result confirm the action the agent reported?
- Can someone stop the run and inspect the evidence?
The list is an editorial design aid, not a description of Oodle AI's feature set. It turns a broad request to “monitor agents” into specific checks a team can assign and review.
The research context
The clip also references Anthropic's investigation of cybersecurity evaluation incidents. The report describes a retrospective review of 141,006 runs that identified three incidents of unauthorised access to third-party infrastructure.
That example illustrates the difference between having records and finding consequential behaviour in them. It does not establish how often such incidents occur in ordinary production deployments.
FAQ
How is AI observability different from application monitoring? Application monitoring measures operational behaviour such as errors and latency. Agent observability also needs evidence about task correctness, tool use and the authority behind an action.
Are logs and traces still useful? Yes. They provide the execution evidence needed to investigate a run. The additional work is identifying which recorded behaviour was incorrect, unexpected or unauthorised.
Is this a standalone full interview? This is a five-minute excerpt from the Agent Accountability documentary. It has its own page so the observability discussion can be watched and read directly.
Guest and further reading
- Gaurav Maheshwari — LinkedIn
- Oodle AI
- Anthropic — Investigating incidents in cybersecurity evaluations
- Watch this excerpt on YouTube
- Agent Accountability — original documentary
Hosted by Himanshu Saxena, Diary of a CTO.