FedLore: אופטימיזציה וזיכרון יעילים ללמידה רפודרטיבית דרך פרויקטציה נמוכת-דרגה של גרדיאנט

תקציר מקורי באנגליתarXiv:2610.01620v1 Announce Type: new Abstract: Federated training of foundation models is constrained by client memory and communication costs. LoRA-based methods reduce these costs through low-rank adapters, but their fixed rank budget can limit adaptation. Gradient low-rank optimization offers greater flexibility, yet independently chosen client subspaces create a problem we term \emph{subspace fragmentation}: local projections interact with data heterogeneity to bias aggregated directions, while aggregation can increase update rank and communication cost. Thus, accurate local gradient compression need not preserve global descent. We propose \texttt{FedLore}, which shares a low-rank optimization basis within each round and refreshes it across rounds. The shared basis enables exact aggre