LLM Probability Concentration: How Alignment Shrinks the Generative Horizon

תקציר מקורי באנגליתarXiv:2506.17871v4 Announce Type: replace-cross Abstract: Despite their impressive capabilities, aligned large language models (LLMs) often generate outputs that lack diversity. What drives this consistency in the generation? We investigate this phenomenon through the lens of probability concentration in the model's output distribution. To quantify it, we use the Branching Factor (BF)--the exponentiated length-averaged entropy of the output distribution, interpreted as the effective number of plausible next steps during generation. Our empirical analysis reveals two key findings: (1) BF often decreases as generation progresses, suggesting that LLMs become more predictable; a controlled intervention indicates that this decline is largely a task-independent property of autoregressive self-co