מתי אופטימיזציה-ללא-קנה-צורה הופכת לבלתי יציבה? חוק זמן רציף עם פגיעה במשקל

תקציר מקורי באנגליתarXiv:2609.09116v1 Announce Type: new Abstract: Normalization renders large parts of neural networks effectively scale invariant, inducing a hidden feedback loop in which learning-rate schedules and weight decay interact through the parameter norm to control the effective step taken by the optimizer. We show that this interaction is governed by an exact discrete-time law: a single scalar quantity captures all schedule and decay forcing, while norm growth induces an opposing geometric self-quenching effect. This yields a sharp boundary that cleanly separates contraction- and expansion-dominated effective learning rate regimes. To understand the underlying mechanism, we provide exact analysis of a fully solved normalized regression model where the dynamics reduce to two dimensions and show t