Dimension-Free Rank Lifting from Random Hyperplane Arrangements

תקציר מקורי באנגליתarXiv:2609.39855v1 Announce Type: new Abstract: We study the width required for a randomly initialized hidden layer of a neural network to achieve rank lifting. Namely, given a dataset $X \in \mathbb{R}^{m \times d}$ of $m$, $d$-dimensional input vectors separated by an angle of at least $\theta$, we consider the random feature matrix $\sigma(XR)$, where $R$ is standard Gaussian. For positively homogeneous nonpolynomial activations, which include sign, Heaviside, ReLU, and ReLU powers among others, we prove that $$n \gtrsim \frac{1}{\theta}\max\left\{m,\log\left(\frac{1}{\delta}\right)\right\}$$ neurons suffice for $\sigma(XR)$ to have full row rank $m$ with probability at least $1-\delta$. This dimension-free bound exponentially improves the previous general-dimensional guarantee for sign