Anti-Periodic Positional Encoding: M\"obius Boundary Conditions Make In-Context Retrieval Reliable

תקציר מקורי באנגליתarXiv:2607.21405v1 Announce Type: new Abstract: M\"obius RoPE is a rotary positional encoding built on the anti-periodic frequency ladder $\theta_i=\pi(2i+1)/N$: every rotation plane advances by an odd multiple of $\pi$ across the training context, so the positional holonomy is $-1$ and the two ends of the sequence are deterministically coupled through a closed-form Dirichlet "dipole"; to our knowledge this is the first anti-periodic boundary condition in positional encoding. We verify the theory numerically to $\sim 10^{-6}$ and pretrain 48 models spanning six 160M-class and three 410M-class arms (2B FineWeb-Edu tokens each; the hybrid arm puts M\"obius frequencies on 25% of heads). Hybrid perplexity is unchanged (29.66 vs. 29.72), but needle-in-a-haystack retrieval becomes reliable: $90.