Attention-Weighted Value Projection for KV-Cache Compression

תקציר מקורי באנגליתarXiv:2604.11501v2 Announce Type: replace Abstract: Rank reduction discards dimensions; quantization keeps them at lower precision. Comparing the two requires a choice of what compression should preserve. For attention values, we study reconstruction of the attention output rather than reconstruction of the values alone. With fixed attention weights, the optimal orthogonal rank-$r$ projection uses the leading eigenvectors of $V^\top\alpha^\top\alpha V$, and its error is exactly the discarded eigenvalue sum. We extend this objective to calibration datasets and grouped query attention, and describe rank allocation under an additive local error budget. We also examine the limits of using local error to predict downstream loss. Historical weight-perturbation experiments favor coefficient round