AI Observability: Practical Guides, Code and Agent Reliability
AI observability work by Soumendra Kumar Sahoo: Hermes and Langfuse instrumentation, the WideScope trace viewer, OpenTelemetry with Python and LLM evaluation. AI observability connects what an AI system did with the evidence needed to debug it and judge its behavior. Traces show retrieval, model calls and tool execution. Evaluation asks whether the result was useful and correct. Cost, latency and failure signals help decide what to change. I’m Soumendra Kumar Sahoo, an AI Observability Architect at PepsiCo. I have worked in enterprise engineering since 2012. My career profile covers my professional history; this page collects my public writing, tools and teaching on observability, evaluation and agent reliability. Adding Langfuse observability to Hermes Agent follows my self-hosted deployment: enabling the plugin, making the Python SDK survive a container redeploy and inspecting model usage and cost. Use it when an agent works but its execution is difficult to explain. The walkthrough includes failures and screenshots from my setup. It is a dated deployment account; version-pinned reproduction and measured improvement are follow-up work, not results established by this article. I create and maintain WideScope, a browser-native viewer for AI-agent traces. Try the viewer with its sample JSON, or inspect the source and fixtures. Use it to explore a captured run without setting up an ingestion service. A trace viewer cannot recover missing instrumentation or establish answer quality by itself. Practical Observability with Python covers logging, metrics, tracing and the OpenTelemetry Collector. The companion repository contains runnable chapter examples; follow the setup instructions for the chapter you are using. Start here if trace propagation or the collector pipeline is unfamiliar. Treat the examples as learning material that needs adaptation to your application’s traffic, privacy and operating constraints. My GitHub carries code and project discussions. My LinkedIn and personal YouTube channel connect the same author to professional updates and videos. OpenOdia is my community education initiative. For a technical conversation or speaking invitation, how to work with me explains what to include in a first message. Start with these three examples
Instrument a running agent: Hermes and Langfuse
Inspect a trace: WideScope
Build the foundations: practical observability with Python
Choose the problem you need to solve
Your question Start here What the material helps explain How does a request keep its identity across services? How distributed tracing works Trace IDs, parent spans and W3C Trace Context, with interactive figures How do I move from dashboards to useful AI signals? Advancing AI observability An introduction to the operating questions behind monitoring What can LLM-as-a-judge cost? The hidden cost of LLM-as-a-judge Evaluation cost and design tradeoffs How does retrieval architecture change a RAG system? The evolution of RAG Retrieval approaches and their design context How should leaders organize AI governance? AI governance playbook Policies, responsibilities and operational controls Connect teaching to implementation
Follow the work