Amplitude launches Agent Analytics
Amplitude's new Agent Analytics combines agent conversation traces with evals to measure task success and how it affects metrics like retention.
With AI agents, more user engagement does not necessarily mean more value. A user can spend five minutes talking to an agent and leave frustrated because it never answered their question. Instead, you have to look at the traces - the actual conversations, responses, and tool calls, to understand whether the agent did its job. That’s why I’m excited to try @Amplitude_HQ's new Agent Analytics. It combines traces with evals that measure the quality of agent interactions, then connects both to what users do next. This way, teams can see: - What the user was trying to accomplish - Where the agent failed - Whether good and bad answers affected KPIs Early results show why this matters: - The Economist (see below) reached 96.9% task success and cut weekly task failures by 84%. - Across 20K+ Amplitude users, a positive first agent experience was associated with 3× higher retention. Check out Agent Analytics here: https://t.co/Eziue3gC6u
This post quotes another post: x.com/i/web/status/2088323335817941328