Blind Spots & Broken Bots: AI Agent Observability

Blind Spots & Broken Bots Getting an AI agent running is one thing. Understanding what it's doing when something goes wrong is another. As agents become more complex, debugging failures gets harder. A failed tool call, an unexpected LLM response, or an agent stuck in a loop might not trigger a traditional alert. Without visibility into what's happening inside the workflow, troubleshooting quickly turns into guesswork. For this Prompt Engineers AI session, we'll explore how to bring observability to AI agents using an open-source, vendor-neutral stack built around LangChain, OpenTelemetry, and the OpenTelemetry Collector. We'll look at how to instrument agents, collect telemetry, and route traces, metrics, and logs into Prometheus, Tempo, Loki, and Grafana. We'll also cover how using the OpenTelemetry Collector helps keep observability backends separate from application code, whether you're running agents in Docker, Kubernetes, or another containerized environment. What we'll cover: Agent tracing: Follow agent execution from LLM interactions through tool calls to better understand behavior and failures. Telemetry collection: Use OpenTelemetry and the Collector to route traces, metrics, and logs. Visualizing agent behavior: Build Grafana views to help identify failures and troubleshoot workflows. Vendor-neutral observability: Understand how to change observability backends at the Collector without modifying agent instrumentation. Starter implementation: Walk away with an open-source boilerplate implementation you can explore, extend, and use as a foundation for your own projects. Who should attend? Platform engineers, SREs, DevOps practitioners, and developers* interested in building or operating AI agents. No prior OpenTelemetry experience is required. Whether you're already running agents or just starting to explore what it takes to operate them reliably, this session will provide a practical…