LLMRouter: Unified Infrastructure for Developing, Evaluating, and Deploying LLM Routers

No single large language model (LLM) is optimal across all queries and budget constraints, making model routing essential for cost-effective LLM deployment. Existing routers span binary quality predictors, cost-aware cascades, graph-based routers, and agentic routers, yet their diverse formalisms and incompatible implementations, coupled with the absence of a standardized evaluation pipeline, hinder fair comparison and further extension. In this paper, we present a unified formulation of LLM routing as a sequential decision process. Under this formulation, a router can be characterized in terms of five types of components: context encoders, model encoders, scoring functions, decision rules, and learning signals. Existing methods can then be organized into three families of single-turn, multi-turn, and personalized routing. Building on this formulation, we develop an automated pipeline that constructs routing supervision by systematically running a pool of candidate models across benchmarks and evaluates routers in terms of both response quality and inference cost under a unified protocol. The resulting benchmark, xRouteBench, spans generic LLM tasks, memory-augmented, vision (image and video), time-series, and personalized routing scenarios.

Introduction. The rapid proliferation of large language models (LLMs) has created a heterogeneous ecosystem of models with widely varying costs and task-specific capabilities, ranging from frontier systems to substantially cheaper open-weight alternatives. Since no single model is optimal across all queries and budget constraints, model routing, which determines which model should handle each query, has become essential for cost-effective LLM deployment. Beyond cost efficiency, routing also matches each query to the candidate model best suited to it and adapts model choice to user-specific preferences (Figure 1).

Discussion / Conclusion. We introduced LLMRouter, a unified framework for LLM routing that casts single-turn, multi-turn, and personalized routing as instances of a common sequential decision process. LLMRouter also provides an automatic pipeline for constructing routing supervision and evaluation for new tasks and candidate pools, the multi-scenario xRouteBench benchmark, and an open-source library that implements more than 16 routers behind a unified interface and supports deployment to real users and multi-agent systems. We hope LLMRouter will serve as a common foundation for developing, evaluating, and deploying LLM routers.