Which provider breaks your tool calls?

See the index
01By the team behind outlines (73M+ downloads)

No
Bad
Actions

Tool calls that parse the first time.

Tool calls that parse, JSON that fits the schema.

Tool calls that parse, JSON that fits the schema.

02They use our technology
  • Liquid AI
  • Hugging Face
  • Doubleword
  • NVIDIA
  • LMU Klinikum
03Evidence

Same weights. Different reliability.

The .txt Tool-call index tracks the tool-call error rate of every provider endpoint on OpenRouter, from OpenRouter's own published figures. Providers serving the same open weights fail very differently: for 34 of 46 models, the worst provider breaks tool calls at least ten times as often as the best; for 11, a hundred times.

A grid of open-weight models by OpenRouter providers, each square coloured by the share of tool calls that come back invalid, from green under 0.1% to dark rust above 10%. Along each model's row the colours swing from green to rust: the same weights fail very differently depending on who serves them.

OpenRouter's published tool-call error rates, October 27 to November 2, 2026: the 25 models with the most providers, and the 42 providers serving at least three of them. 368 provider-model pairs with 1,000 or more tool calls a day. The 34 and 11 count all 46 models served by at least two such providers.

04What we do

Valid tool calls and JSON, as the model writes them.

.txt builds structured generation into the inference engine. At every step, the model can only write what your tools and schemas allow. We work with inference providers and teams that ship models on devices.

Play it on your own model. 

The Z800 runs mock calls. Tell us your model and your tools, and we'll run the real ones.