Boxiao Du, Boning Huangfu, Yizhou Luo, , and 5 more authors
arXiv preprint arXiv:2605.16867, 2026
GoodServe is a goodput-optimized serving system for agentic LLM inference over heterogeneous GPU resources. It combines predict-and-rectify routing, just-enough instance selection, SLO-risk monitoring, and request migration to improve goodput for latency-sensitive agentic workloads.
@article{goodserve2026,
title = {GoodServe: Towards High-Goodput Serving of Agentic LLM Inferences over Heterogeneous Resources},
author = {Du, Boxiao and Huangfu, Boning and Luo, Yizhou and Chen, Chen and Li, Zijun and Yu, Minchen and Fan, Xiaoyi and Guo, Minyi},
journal = {arXiv preprint arXiv:2605.16867},
year = {2026},
arxiv = {2605.16867},
doi = {10.48550/arXiv.2605.16867}
}