סיבוב סטוכסטי באינפרנסיה של תרגומן רכזי בעלי זיכרון נמוך: מחקר על התאמה של זיכרון רכזי סטוכסטי

תקציר מקורי באנגליתarXiv:2610.01889v2 Announce Type: replace-cross Abstract: Should low-precision transformer inference use stochastic rounding (SR) or round-to-nearest (RN)? The answer depends on where in the network you look. We isolate this effect by holding the numerical format fixed and varying only the rounding rule at individual operation sites. To enable experiments at freely chosen precisions, we extend the PRISM vectorized rounding library to arbitrary virtual precision via a variable-precision stochastic rounding (VPSR) algorithm, proving that the rounding decision is evaluated exactly in hardware floating point. We develop two analyses providing complementary insight into this site-level trade-off. First, a probabilistic forward-error bound for linear projections shows that SR's error envelope gr