בחירת גרדיאנט דסקנט עם ReLU: הגבול החופשי מהאירועים

תקציר מקורי באנגליתarXiv:2608.30960v2 Announce Type: replace Abstract: Gradient flow is widely used as a continuous-time surrogate for gradient descent, but state convergence does not imply convergence of differentiated training maps in nonsmooth networks. We characterize the fixed-horizon, vanishing-step limit of exact automatic differentiation through hard-ReLU gradient descent. Under a stable finite itinerary of separated, same-direction transverse activation events, gradient-descent states converge at first order to the corresponding piecewise-smooth gradient flow, while the exact derivative of every nonresonant discrete program converges to an event-free regional propagator. The true flow derivative instead interleaves classical saltation matrices that encode event-time sensitivity. For globally convex