כתבה
arXiv cs.LG ·
למידה מפתרונות עצמם: חיסול קרדיט עצמי להגדרת קרדיט למדריכי RLVR
Learning from Own Solutions: Self-Conditioned Credit Assignment for Reinforcement Learning with Verifiable Rewards
אורחות הלמידה של רכיבי RLVR נשפרו על ידי חיסול קרדיט עצמי, כולל חיסול קרדיט של GPT-5
תקציר מקורי באנגליתarXiv:2606.18810v2 Announce Type: replace Abstract: Reinforcement learning with verifiable rewards (RLVR) has driven substantial progress in training LLMs for reasoning tasks, but representative methods such as GRPO assign uniform credit across all tokens, wasting gradient on routine tokens while under-crediting pivotal reasoning steps. Existing token-level credit assignment methods require resources beyond the model's own rollouts. GRPO variants rely on process reward models or ground-truth answers. Knowledge distillation assigns credit through per-token divergence but requires external teachers (On-Policy Distillation) or privileged information (On-Policy Self Distillation). However, these dependencies limit applicability in the pure RLVR setting. We observe that conditioning the model o
קרא במקור המקורי
arxiv.org
פתח כתבה מקורית