10 open-source projects for monitoring what AI agents do
Using a GPT-6 Astra workflow as a hook, the post lists 10 open-source observability and evaluation projects such as Langfuse, Phoenix, AgentOps and DeepEval, plus three suggested stacks for execution, quality and production.
this GPT-6 Astra workflow is getting f…cking unfair 10 open-source GitHub projects for seeing what your AI agents doing while you sleep 01 Langfuse ▸ https://github.com/langfuse/langfuse → traces + evals + metrics 02 Arize Phoenix ▸ https://github.com/Arize-ai/phoenix → inspect LLM + agent execution TRACE EVERY MOVE 03 AgentOps ▸ https://github.com/AgentOps-AI/agentops → monitor multi-agent sessions 04 OpenLLMetry ▸ https://github.com/traceloop/openllmetry → OpenTelemetry for LLM apps 05 OpenLIT ▸ https://github.com/openlit/openlit → traces + costs + performance WATCH THE RUNTIME 06 Pydantic Logfire ▸ https://github.com/pydantic/logfire → application-level observability 07 Helicone ▸ https://github.com/Helicone/helicone → monitor LLM requests + latency + spend TEST THE AGENTS 08 DeepEval ▸ https://github.com/confident-ai/deepeval → evaluate agent behavior 09 Ragas ▸ https://github.com/explodinggradients/ragas → measure retrieval + response quality CATCH FAILURES 10 Sentry ▸ https://github.com/getsentry/sentry → errors + performance + debugging the control loop: agent → tool call → trace → metric → eval → failure → inspect → retry 3 views I’d actually build: execution: Langfuse → AgentOps → OpenLLMetry quality: Phoenix → DeepEval → Ragas production: OpenLIT → Logfire → Sentry and this is where it gets interesting: an autonomous agent can make hundreds of decisions before you see one final answer if you can’t inspect the decisions, tools, failures, and retries you don’t really control the agent ⭣
This post quotes another post: x.com/lummox_eth/status/2100288151897981255