back

BoundaryRouter (arXiv:2605.07180): Training-Free LLM-vs-Agent Router Cuts Inference Time 60.6%; Introduces RouteBench

today 17:05

BoundaryRouter (May 8, 2026) is a training-free framework that executes both direct LLM inference and a full agent on a shared seed set, building an experience memory for routing decisions at inference time. On RouteBench—a new benchmark with in-domain, paraphrased, and out-of-domain splits—it reduces inference time 60.6% versus full-agent execution while outperforming direct LLM inference by 28.6%, prompt-based routing by 37.9%, and retrieval-only routing by 8.2%.

Citations