AveryChing: verifiable records to hold AI agents accountable
The author argues the nearer risk is AI agents being scapegoated for human failures, and calls for a tamper-resistant ledger of human instructions, delegated authority and agent actions, naming Jev as accelerating the challenge.
Contrarian opinion: While the current fear of AI ending the world is top of mind for everyone, I think a more near-term problem is the scapegoating AI agents for human failures. The headlines will come: * Drone attack kills friendlies -> AI targeting failed * Bad options trade costs firm billions -> AI made the wrong analysis * Surgery goes wrong -> AI misidentified a tumor * Company pays $50M to a fake vendor -> AI systems approved it Humans are experts at avoiding accountability. And AI is imperfect in achieving its goals or even following them. True accountability requires a tamper-resistant, verifiable record of the chain of events. The legal system does this through a lengthy discovery process from a blurry timeline, but agents and human interactions are all digital and can be inexpensively captured in real time with high precision. AI capabilities and cost effectiveness are moving faster than any technology we have ever seen. New systems like JEV and rapid advancements in retaining model quality with dramatically less hardware are accelerating this challenge faster than our defenses. Humans are responsible for their intentions and the authority they delegate. Agents implement and execute those intentions. And there is a gray area of human/AI interaction: when to involve a human, when a human has given permissions (e.g. one-shot or blanket), and how to retrace these interactions whether the outcomes are terrific or terrible. What did the human ask for and approve? What authority did the agent have? What information did it receive? What actions did it take? When was a human involved? Where did human language leave ambiguity that should have been clarified? And can any of that history be altered after the fact? Immutable ledgering, powerful auditing tools, and deeply integrated human/AI interaction are necessary for humanity to achieve its full potential. @Aptos @shelbyserves @DecibelTrade are all being built for this imminent reality.
This post quotes another post: x.com/aptos/status/2101703109118701676