כתבה
Simon Willison ·
ttok 1.0
ttok 1.0 יצא לאור, עם תקן תזמון גרסה 5 של GPT כברירת מחדל
תקציר מקורי באנגלית<p><strong>Release:</strong> <a href="https://github.com/simonw/ttok/releases/tag/1.0">ttok 1.0</a></p> <p>I released <a href="https://simonwillison.net/2026/Oct/8/ttok/">ttok 0.4</a>, ran <code>uv tool upgrade ttok</code>, piped a file into the new version... and realized that it was defaulting to the GPT-4 tokenizer when it should very clearly default to GPT-5/GPT-6 instead!</p> <p>I figured switching the default was a reasonable excuse to finally ship a 1.0.</p> <p>OpenAI haven't actually confirmed that GPT-6 uses the same tokenizer as the GPT-5 family yet - there's an <a href="https://github.com/openai/tiktoken/issues/608">angry issue about it</a> - but I found <a href="https://github.com/williamliu-ai/token-count-compare/commit/8d8a2178538bb37beb548fc37378135e4ff4b0c8">this commit</a>
קרא במקור המקורי
simonwillison.net
פתח כתבה מקורית