Aria tests 21 routes to back its GPT-6 Astra labeling
Aria tested all 21 of its routes for these models against OpenRouter references, saying its Astra routes scored the same 12/12 on objective tasks at about 70% lower cost, while acknowledging cheaper routes vary.
We've seen the community's questions about whether Aria delivers the models it advertises. Here's our direct response. Aria offers these routes as GPT-6 Astra, Claude Fable 5.1 and Claude Opus 5.5, not as substitutes for older models. We are responsible for verifying that those labels accurately reflect what users receive. We tested all 21 available Aria routes for these models alongside OpenRouter references pinned to OpenAI and Anthropic. We then expanded the comparison with objective tasks at Low and High reasoning. The objective suite covered mathematics, Python code semantics, shortest-path reasoning, structured data processing, long-context retrieval and optimization. With the same 12 tasks and matched output limits: • OpenRouter's OpenAI-pinned Astra: 12/12 at Low, 12/12 at High. • Aria 0.44x: 12/12 at Low, 12/12 at High. • Aria 0.22x: 12/12 at Low, 12/12 at High. • Aria's cheapest tested route: 12/12 at Low, with all 11 completed High answers correct and one request incomplete. Some requests required retries. These are completed-answer scores, not a claim of identical reliability across routes. For the 0.22x route, those completed answers cost: • Low: $0.07164 on Aria vs $0.24763 on OpenRouter. • High: $0.09796 on Aria vs $0.31348 on OpenRouter. The same task scores, at approximately 69–71% lower cost. We also found differences on some cheaper routes, including at Low reasoning. We're investigating the routing and reasoning configuration behind those differences. We haven't established the cause, and we won't describe High as a universal fix: both 0.22x and 0.44x already scored full marks at Low. On the specific cutoff allegation: with OpenRouter pinned to OpenAI and reasoning set to High, Astra answered "June 2024" in two of three responses. The third gave no precise date. That disputed answer isn't unique to Aria and cannot, by itself, prove model substitution. Matching task scores don't independently certify model identity either. Our investigation needs to address both performance and the routing evidence behind the model labels. The report includes exact prompts, settings, answers, scoring methods and 90 OpenRouter generation receipts: https://entelic.io/reports/model-comparison/ TL;DR: Aria's 0.22x route matched the official-provider reference score on our tests for about 70% less. The "old date = fake model" claim isn't proof. The results are public, and we're checking the remaining route differences. You deserve the model you pay for. You deserve the model you selected. We stand behind that standard, and any route we cannot verify should not carry that label.