Report
A Shape-Adaptive Architecture with Disaggregated Quantization for Efficient LLM Serving
| Title: | A Shape-Adaptive Architecture with Disaggregated Quantization for Efficient LLM Serving |
|---|---|
| Authors: | Guo, Cong, Wei, Chiyue, Duan, Bowen, Shan, Haoxuan, Morris III, Benjamin F., He, Yintao, |
| Publication Year: | 2026 |
| Subject Terms: | Hardware Architecture, C.1.4, C.3, I.2.7 |
| Description: | Large language models (LLMs) have become the backbone of modern AI applications, but pose significant challenges for efficient inference. Their autoregressive generation divides execution into two phases: prefill, dominated by large GEMMs, and decoding, dominated by small GEMVs. Modern serving systems further introduce complexity through continuous batching and prefill-decoding disaggregation, leading to dynamic workloads and phase separation. However, existing accelerators remain poorly aligned with these system-level behaviors, resulting in inefficiencies in LLM serving. In this work, we present DynaCore, a unified architecture for efficient LLM serving via system-architecture co-design. We observe that the compute tile a systolic array executes, its Minimum Efficient Unit (MEU), spans all three GEMM dimensions. DynaCore reshapes the MEU along all three: spatially it trades array width against height asymmetrically, raising weight delivery while leaving the input path untouched, and temporally Split-K maps the reduction onto the array, folding partial sums through the interconnect the array already has. To exploit phase separation, we further propose disaggregated quantization, applying dual-side quantization to prefill and weight-only quantization to decoding, with an inner-product mixed-precision datapath that keeps output width invariant to precision. A runtime scheduling framework then selects an MEU per batch. Evaluation with real-world serving traces shows that DynaCore substantially reduces service-level latency over quantization and reconfigurable accelerators, improving TTFT by 3.50x and 2.97x and TPOT by 36.55x and 8.02x, respectively. 14 pages, 14 figures, 4 tables |
| Document Type: | Working Paper |
| DOI: | 10.48550/arXiv.2610.07443 |
| Access URL: | https://arxiv.org/abs/2610.07443 |
| Accession Number: | edsarx.2610.07443 |
| Database: | arXiv |
| DOI: | 10.48550/arXiv.2610.07443 |
|---|