כתבה
arXiv cs.CL ·
JEV נגד LLMs: דיוק, עלות וכיול
JEV versus LLMs: Accuracy, Cost and Calibration on Seven Political Science Replications
JEV הוא מודל חדש שמתחרה במודלים LLMs. הוא מציע יתרון בעלות ובמהירות, אך האם הוא גם מדויק? המחקר משווה JEV ל-GPT-6 Luna ו-Qwen3.8-27B.
תקציר מקורי באנגליתarXiv:2610.06625v2 Announce Type: replace Abstract: Large language models (LLMs) annotate and scale political text or constructs by generating text tokens. A new class of models, which TypeSafe markets as "System One" models, instead returns decisions and probability distributions across a user-supplied fixed answer set. A commercial model, JEV, is advertised as having a dramatic cost and speed advantage over traditional LLMs along with better calibrated decisions. As such, it might be useful for social scientists looking to quickly and cost-effectively annotate or scale large corpora of text and have a reliable indicator of a classifier's uncertainty. Yet, the accuracy of these claims and the broader model accuracy in social science text-based tasks are not yet established. In this paper,
קרא במקור המקורי
arxiv.org
פתח כתבה מקורית