כתבה
arXiv cs.LG ·
TeDiServe: שירות גבוה להגעה ל-SLO עבור דיפוזיה של מודלי שפה
TeDiServe: High SLO Attainment Serving for Diffusion Language Models
TeDiServe השיגה הגברת של עד 56.6% בהגעה ל-SLO והפחתת עד 46% בעיכוב תקשורת בין-שלבית.
תקציר מקורי באנגליתarXiv:2606.29094v2 Announce Type: replace Abstract: Diffusion language models (DLMs) have recently emerged as a promising alternative to conventional autoregressive language models. By generating multiple tokens in parallel during each denoising step, they offer higher inference throughput while maintaining competitive quality. However, realizing these throughput gains while meeting latency SLOs in a serving system requires addressing challenges introduced by DLMs' unique characteristics. These include navigating the speed-quality tradeoff created by confidence-based denoising, choosing appropriate parallelization levels across model instances under fluctuating load, and coordinating approximate KV caching mechanisms that introduce non-uniform per-step costs. To address these challenges, w
קרא במקור המקורי
arxiv.org
פתח כתבה מקורית