Test: Non-English Prompts Cost More Tokens in Most LLMs

A Habr test compared 11 tokenizers on parallel Russian and English texts. Russian text needs 2.96x more tokens on Claude Opus 5 and 1.19x on GPT-5.x, while YandexGPT 5 Lite and GigaChat 3 use fewer tokens for Russian than for English.

DEV Community

Open source

Test: Non-English Prompts Cost More Tokens in Most LLMs

Claude Opus 5: Russian text takes 2.96x tokens, a technical article 872 vs 327
GPT-5.x (o200k) at 1.19x, Gemini 3.7 Flash at 1.20x, DeepSeek V3/V4 at 1.38x
YandexGPT 5 Lite (0.91x) and GigaChat 3 (0.96x) are cheaper in Russian than English
At $5 per million tokens, Russian on Claude Opus 5 costs 3.9x more than on GPT-5.5

#Anthropic#OpenAI#Google#Yandex

Test: Non-English Prompts Cost More Tokens in Most LLMs

New Benchmark Tests 24 LLMs Against Human Writers on 475 Prompts

Test: 10 of 12 new LLMs recall earnings outcomes from just a name and a date

Opus 5.5 vs GPT-6 Sol: cost per task, not per token

Grok 4.7 keeps $2/$6 token pricing but doubles cost per task