Login

u/silksong_when

r/FastAPI r/Python r/Observability
Observing LLM Applications with OpenTelemetry
▲ 23 r/Observability+1 crossposts

Observing LLM Applications with OpenTelemetry

Hi guys,

I have written about my experience exploring the state of LLM integrations for observing agentic workflows and LLM integrations using OpenTelemetry.

While the guide uses SigNoz as the observability backend, you can use whatever backend you prefer, since it's all OTel-backed.

In the blog itself, I have gone over:

  • why you need to observe LLMs
  • brief intro on OpenTelemetry
  • dissecting the companion NBA Agent I'd prepared using the OpenAI Agents SDK, which runs in an agentic loop utilizing guardrails and session persistence
  • shortcomings in the instrumentation libraries to be aware of
  • developments ongoing within the OpenTelemetry dev communities focusing on LLM observability
signoz.io
u/silksong_when — 21 days ago
▲ 37 r/FastAPI+1 crossposts

Hi Pythonistas, I recently revamped our article on Implementing OpenTelemetry in FastAPI Projects in a practical manner, which was originally written in 2024 and needed a fresh coat of paint.

The article covers auto-instrumentation, manual spans, visualizing metrics and how observability lets you understand how your web apps behave.
I've also included some advanced tips, such as, selective error tracking, and wrapping dependency functions to capture any operations within the `yield` scope.

Since a lot of the concepts discussed here are independent of the FastAPI framework, any developer working with Python can probably find something of use here.

Finally, I hope this write up helps some folks become familiar with OpenTelemetry and observability.
Any feedback would be much appreciated, also curious to understand what problems you face with monitoring your web apps, be it FastAPI or any other web framework.

---

On a personal note, when implementing OpenTelemetry in my previous job, I went in semi-blind and relied on agents to guide me, and then spend a good week dealing with the various issues that popped up along the way...

u/silksong_when — 2 months ago