כתבה
arXiv cs.LG ·
Two-Level Softmax Sampling: תיקון שיבושים בסייסטמינג
Two-Level Softmax Sampling Done Right: Correcting Bias from Size Imbalance and Dispersion
ניסוי חדש: Two-Level Softmax Sampling Done Right - תיקון שיבושים בסייסטמינג. פיתוח שני שיטות סייסטמינג חדשות, S-2LS ו-SD-2LS, שמציעות דיוק טוב יותר ומעט עלויות חישוביות.
תקציר מקורי באנגליתarXiv:2610.10483v1 Announce Type: new Abstract: Sampling from a softmax distribution is a fundamental operation in machine learning, but its linear complexity in the number of items makes exact sampling impractical at scale. Two-level softmax (2LS) sampling is a popular alternative enabling sublinear-time sampling. Assuming items are partitioned into clusters, 2LS first samples a cluster and then an item within it. In this paper, we show that, despite its advantages, 2LS introduces systematic and undesirable sampling biases, which arise from misweighting clusters by ignoring both cluster size imbalance and intra-cluster similarity dispersion. We propose two sampling methods, Size-Corrected 2LS (S-2LS) and Size- and Dispersion-Corrected 2LS (SD-2LS), which correct these biases and provide p
קרא במקור המקורי
arxiv.org
פתח כתבה מקורית