כתבה
arXiv cs.LG ·
ManifoldFlow: רפלקסיה של שכבת Stiefel SPD-Relaxed
ManifoldFlow: SPD-Relaxed Stiefel Layers with Learnable Singular Spectrum
מוצגת ManifoldFlow, רפלקסיה של שכבת Stiefel שבה הספקטרום הסינגולרי הוא למד.
תקציר מקורי באנגליתarXiv:2607.04535v2 Announce Type: replace Abstract: Orthogonal and Stiefel layers give neural weights exact spectral control, but they also impose a strong modeling constraint: all represented singular values are fixed at one. Many settings that benefit from an orthonormal basis still need direction-dependent attenuation or amplification. We introduce ManifoldFlow, a minimal relaxation of a fixed-spectrum Stiefel layer that keeps the basis on the Stiefel manifold while learning a bounded positive spectrum through W = Q S^{1/2}, with Q^T Q = I and S positive definite. Since W^T W = S, the eigenvalues of S are exactly the squared singular values of the realized weight, making eigenvalue clipping a direct singular-value control mechanism. Across paired sequence, tabular, and image experiments
קרא במקור המקורי
arxiv.org
פתח כתבה מקורית