כתבה
arXiv cs.CL ·
כאשר Rank עולה כשה-LLMs נפגעים
When Rank Rises as LLMs Degrade
במחקר חדש נמצא שכאשר דגמי LLM נפגעים, רמת Rank עולה, ולא יורדת. זה יכול לגרום לבעיות בשימוש בדגמים אלה.
תקציר מקורי באנגליתarXiv:2610.09647v1 Announce Type: cross Abstract: Post-training adapts language models in non-stationary environments. Practitioners monitor representation health with RankMe and related spectral statistics, often assuming that rank falls when representations degrade. We show that this assumption is unsafe for LLM post-training. In a controlled study of Qwen3-0.6B with four degradation modes and three seeds, data duplication worsens held-out loss by 75% relative to healthy while increasing both original and centred RankMe; the latter changes by 13.5 pooled standard deviations. Covariance effective rank rises to nearly twice its healthy value. This failure is spectral dispersion rather than collapse, so a one-sided monitor rates the worst checkpoint as the healthiest. By contrast, a learnin
קרא במקור המקורי
arxiv.org
פתח כתבה מקורית