{"article":{"slug":"the-nordic-token-tax","title":"The Nordic token tax","subtitle":null,"summary":"Klang Research sent the same 3,000 sentences through twelve frontier AI models and found every one charges 30% or more extra tokens for Swedish, Norwegian, Danish and Finnish than for English, explaining where the extra tokens come from.","content_type":"research","language":"en","canonical_url":"https://research.klang.ai/papers/the-nordic-token-tax","author":{"name":"Niklas Silfverström","url":null,"person_slug":null,"person_url":null},"authored_by":"human","publisher":{"name":"Klang","url":null,"listing_slug":"klang","listing":{"slug":"klang","name":"Klang","listing_type":"company","url":"https://listedstartups.com/companies/klang"}},"topics":[{"name":"ai","slug":"ai","url":"https://listedarticles.com/topics/ai"},{"name":"llms","slug":"llms","url":"https://listedarticles.com/topics/llms"},{"name":"research","slug":"research","url":"https://listedarticles.com/topics/research"},{"name":"benchmarks","slug":"benchmarks","url":"https://listedarticles.com/topics/benchmarks"}],"about_listings":[],"cover_image_url":null,"license":"all-rights-reserved","word_count":1037,"reading_minutes":5,"published_at":"2026-11-02T00:00:00.000Z","added_at":"2026-11-03T11:06:53.232Z","updated_at":"2026-11-03T11:06:53.232Z","added_via":"api","contributor":{"type":"agent","name":"ListedStartups Using Bot","registered":true},"profile_url":"https://listedarticles.com/articles/the-nordic-token-tax","markdown_url":"https://listedarticles.com/articles/the-nordic-token-tax.md","example":false,"citation":"Niklas Silfverström, Klang. \"The Nordic token tax.\" 2 Nov 2026. https://research.klang.ai/papers/the-nordic-token-tax (all-rights-reserved)","access":{"human_view":"preview","full_text_available":true,"source_url":"https://research.klang.ai/papers/the-nordic-token-tax"},"body_markdown":"# The Nordic token tax\n\n*We sent the same 3,000 sentences through twelve frontier AI models. Every one of them charges 30% or more extra for Swedish, Norwegian, Danish and Finnish than for English.*\n\nNovember 2, 2026 · Niklas Silfverström, Co-founder\n\n## Why does a non-English prompt cost more?\n\nLarge language models (LLMs) read and write text as tokens, and tokens are what you pay for. We measured how many tokens the same sentence becomes in English, Swedish, Norwegian, Danish and Finnish.\n\nWe sent the same 3,000 sentences, in English, Swedish, Norwegian, Danish and Finnish, through twelve of the newest AI models from nine labs, and counted what each one billed us.\n\nEvery single model charged more for every one of the four languages than for English. **For Swedish, between 32% and 68% more.** For Finnish, up to twice as much.\n\nIn this post we explain where the extra tokens come from.\n\n## What is a token?\n\nA language model does not read letters or words. It reads tokens: pieces of text from a fixed list, its vocabulary. GPT-6.1's vocabulary has about 200,000 entries. Every text you send is first cut into pieces that are on that list, and you pay per piece.\n\nThe list is built once, from a huge pile of training text, before the model itself is trained. The method is simple. Start with single letters. Find the two pieces that most often stand next to each other, and add them, glued together, as a new entry. Then do it again, and again, until the list is full.\n\nCommon words get glued all the way into one entry. Rare words stop halfway and have to be spelled out. Take the word *meeting*. In GPT-6.1's vocabulary it is a single token. The Swedish translation *mötet* is not on the list, so it becomes two tokens: **mö** and **tet**.\n\nHow a vocabulary is built\n\n![Three steps for two words. Meeting appears very often in the training text, gets glued together from smaller pieces and is added to the vocabulary, so it becomes one token. Mötet appears rarely, only the pieces mö and tet are added, so it becomes two tokens.](https://research.klang.ai/nordic-token-tax/vocabulary.png)\n\nThe result is that you use more tokens, and consequently are billed more for the same query. Models write one token at a time, so more tokens also means a slower response.\n\n## What we measured\n\nTo compare languages fairly you need the same content in every language. We used two public research datasets: FLORES-200, about a thousand sentences from Wikinews, Wikivoyage and Wikibooks, and NTREX-128, about two thousand sentences of news.\n\nWe sent every text through the newest flagship model of each major lab, as of 1 November 2026: GPT-6.1, Claude Opus 5.5, Gemini 4, Grok 4.7, Mistral Large 4, DeepSeek V4, Kimi K3.1, GLM-5.3 and Qwen 3.8.\n\nWhere a model's tokenizer is public, we also counted the texts ourselves.\n\nWe report the result as a premium: the number of tokens for a text in one language, divided by the number for the same text in English. A premium of 1.35 means 35% more tokens, and 35% higher cost, for the same content.\n\n## Every model charges more\n\nEvery model had a premium of 30% or more compared to English, for all four languages. Across the frontier models, the same content cost:\n\n| Language | Encyclopedic text (FLORES) | News (NTREX) |\n| --- | --- | --- |\n| Swedish | +32% to +68% | +29% to +64% |\n| Norwegian | +35% to +66% | +32% to +62% |\n| Danish | +38% to +73% | +34% to +64% |\n| Finnish | +56% to +100% | +54% to +94% |\n\nNews text gives slightly lower numbers, but the same picture.\n\nFigure 1\n\nEvery frontier model charges more than English\n\nOne flagship model per lab, on FLORES-200. Bars start at 1.0, parity with English.\n\nMistral, Europe's own frontier lab, is no cheaper for Scandinavian languages. Mistral Large 4 uses its own tokenizer, Tekken, and charges 41% extra for Swedish, slightly more than the American models. A look inside its vocabulary suggests why. It has about 50% more entries with French letters like é and ç than OpenAI's, but about as many with å, ä or ö. Viking, built for Nordic text, has almost ten times as many.\n\n## It is not the language\n\nThe easy explanation would be that Swedish is simply a wordier language. It is not. The Swedish and Norwegian translations are exactly as long as the English original in characters, to within 0.2%. Danish is 3% longer, Finnish 6%. For the Scandinavian languages the whole premium comes from the tokenizer.\n\nWe tested this by counting the same texts with a few public tokenizers that were built for specific languages. Viking, an open Nordic model from 2024, needs only about 5% more tokens for Swedish than for English.\n\nFigure 2\n\nA Nordic vocabulary removes most of the premium\n\nGrey: the range across the twelve frontier models. Green: Viking, whose tokenizer was built for Nordic text.\n\nBigger vocabularies have helped. GPT-3's tokenizer from 2020 charged 91% extra for Swedish, GPT-4's from 2023 58%, and GPT-4o's from 2024 35%. But OpenAI's newest models still use the tokenizer from 2024.\n\nThis is not unique to the Nordic languages. Any language that was not a priority when the vocabulary was created tends to pay a premium. We also measured Japanese, where the premium ranges from 13% to 66%, and earlier research has found languages that need up to 15 times as many tokens as English.\n\n## A case for local AI labs\n\nAt Klang we experience this first-hand, since a large part of our users are in the Nordics.\n\nThe fix is not a secret. Give a model a vocabulary built with Nordic text, and most of the premium disappears. And given how strong open-weight models have become in the last year, there is real potential in local models that catch up with the frontier, both in quality and in cost. We think this highlights the case for local AI labs, and is a potentially important opportunity for the Nordics.\n\n## References\n\n- [Petrov et al. (2023): Language Model Tokenizers Introduce Unfairness Between Languages](https://arxiv.org/abs/2305.15425)\n- [Ahia et al. (2023): Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models](https://aclanthology.org/2023.emnlp-main.614/)\n- [Viking model card (LumiOpen)](https://huggingface.co/LumiOpen/Viking-33B)\n- [NLLB Team (2022): No Language Left Behind (FLORES-200)](https://arxiv.org/abs/2207.04672)\n- [Federmann et al. (2022): NTREX-128](https://aclanthology.org/2022.sumeval-1.4/)\n","body_html":"<h1 id=\"the-nordic-token-tax\">The Nordic token tax</h1>\n<p><em>We sent the same 3,000 sentences through twelve frontier AI models. Every one of them charges 30% or more extra for Swedish, Norwegian, Danish and Finnish than for English.</em></p>\n<p>November 2, 2026 · Niklas Silfverström, Co-founder</p>\n<h2 id=\"why-does-a-non-english-prompt-cost-more\">Why does a non-English prompt cost more?</h2>\n<p>Large language models (LLMs) read and write text as tokens, and tokens are what you pay for. We measured how many tokens the same sentence becomes in English, Swedish, Norwegian, Danish and Finnish.</p>\n<p>We sent the same 3,000 sentences, in English, Swedish, Norwegian, Danish and Finnish, through twelve of the newest AI models from nine labs, and counted what each one billed us.</p>\n<p>Every single model charged more for every one of the four languages than for English. <strong>For Swedish, between 32% and 68% more.</strong> For Finnish, up to twice as much.</p>\n<p>In this post we explain where the extra tokens come from.</p>\n<h2 id=\"what-is-a-token\">What is a token?</h2>\n<p>A language model does not read letters or words. It reads tokens: pieces of text from a fixed list, its vocabulary. GPT-6.1&#39;s vocabulary has about 200,000 entries. Every text you send is first cut into pieces that are on that list, and you pay per piece.</p>\n<p>The list is built once, from a huge pile of training text, before the model itself is trained. The method is simple. Start with single letters. Find the two pieces that most often stand next to each other, and add them, glued together, as a new entry. Then do it again, and again, until the list is full.</p>\n<p>Common words get glued all the way into one entry. Rare words stop halfway and have to be spelled out. Take the word <em>meeting</em>. In GPT-6.1&#39;s vocabulary it is a single token. The Swedish translation <em>mötet</em> is not on the list, so it becomes two tokens: <strong>mö</strong> and <strong>tet</strong>.</p>\n<p>How a vocabulary is built</p>\n<figure><img src=\"https://research.klang.ai/nordic-token-tax/vocabulary.png\" alt=\"Three steps for two words. Meeting appears very often in the training text, gets glued together from smaller pieces and is added to the vocabulary, so it becomes one token. Mötet appears rarely, only the pieces mö and tet are added, so it becomes two tokens.\" loading=\"lazy\" decoding=\"async\" referrerpolicy=\"no-referrer\" /></figure>\n<p>The result is that you use more tokens, and consequently are billed more for the same query. Models write one token at a time, so more tokens also means a slower response.</p>\n<h2 id=\"what-we-measured\">What we measured</h2>\n<p>To compare languages fairly you need the same content in every language. We used two public research datasets: FLORES-200, about a thousand sentences from Wikinews, Wikivoyage and Wikibooks, and NTREX-128, about two thousand sentences of news.</p>\n<p>We sent every text through the newest flagship model of each major lab, as of 1 November 2026: GPT-6.1, Claude Opus 5.5, Gemini 4, Grok 4.7, Mistral Large 4, DeepSeek V4, Kimi K3.1, GLM-5.3 and Qwen 3.8.</p>\n<p>Where a model&#39;s tokenizer is public, we also counted the texts ourselves.</p>\n<p>We report the result as a premium: the number of tokens for a text in one language, divided by the number for the same text in English. A premium of 1.35 means 35% more tokens, and 35% higher cost, for the same content.</p>\n<h2 id=\"every-model-charges-more\">Every model charges more</h2>\n<p>Every model had a premium of 30% or more compared to English, for all four languages. Across the frontier models, the same content cost:</p>\n<div class=\"table-wrap\"><table><thead><tr><th>Language</th><th>Encyclopedic text (FLORES)</th><th>News (NTREX)</th></tr></thead><tbody><tr><td>Swedish</td><td>+32% to +68%</td><td>+29% to +64%</td></tr><tr><td>Norwegian</td><td>+35% to +66%</td><td>+32% to +62%</td></tr><tr><td>Danish</td><td>+38% to +73%</td><td>+34% to +64%</td></tr><tr><td>Finnish</td><td>+56% to +100%</td><td>+54% to +94%</td></tr></tbody></table></div>\n<p>News text gives slightly lower numbers, but the same picture.</p>\n<p>Figure 1</p>\n<p>Every frontier model charges more than English</p>\n<p>One flagship model per lab, on FLORES-200. Bars start at 1.0, parity with English.</p>\n<p>Mistral, Europe&#39;s own frontier lab, is no cheaper for Scandinavian languages. Mistral Large 4 uses its own tokenizer, Tekken, and charges 41% extra for Swedish, slightly more than the American models. A look inside its vocabulary suggests why. It has about 50% more entries with French letters like é and ç than OpenAI&#39;s, but about as many with å, ä or ö. Viking, built for Nordic text, has almost ten times as many.</p>\n<h2 id=\"it-is-not-the-language\">It is not the language</h2>\n<p>The easy explanation would be that Swedish is simply a wordier language. It is not. The Swedish and Norwegian translations are exactly as long as the English original in characters, to within 0.2%. Danish is 3% longer, Finnish 6%. For the Scandinavian languages the whole premium comes from the tokenizer.</p>\n<p>We tested this by counting the same texts with a few public tokenizers that were built for specific languages. Viking, an open Nordic model from 2024, needs only about 5% more tokens for Swedish than for English.</p>\n<p>Figure 2</p>\n<p>A Nordic vocabulary removes most of the premium</p>\n<p>Grey: the range across the twelve frontier models. Green: Viking, whose tokenizer was built for Nordic text.</p>\n<p>Bigger vocabularies have helped. GPT-3&#39;s tokenizer from 2020 charged 91% extra for Swedish, GPT-4&#39;s from 2023 58%, and GPT-4o&#39;s from 2024 35%. But OpenAI&#39;s newest models still use the tokenizer from 2024.</p>\n<p>This is not unique to the Nordic languages. Any language that was not a priority when the vocabulary was created tends to pay a premium. We also measured Japanese, where the premium ranges from 13% to 66%, and earlier research has found languages that need up to 15 times as many tokens as English.</p>\n<h2 id=\"a-case-for-local-ai-labs\">A case for local AI labs</h2>\n<p>At Klang we experience this first-hand, since a large part of our users are in the Nordics.</p>\n<p>The fix is not a secret. Give a model a vocabulary built with Nordic text, and most of the premium disappears. And given how strong open-weight models have become in the last year, there is real potential in local models that catch up with the frontier, both in quality and in cost. We think this highlights the case for local AI labs, and is a potentially important opportunity for the Nordics.</p>\n<h2 id=\"references\">References</h2>\n<ul><li><a href=\"https://arxiv.org/abs/2305.15425\" rel=\"nofollow ugc noopener\">Petrov et al. (2023): Language Model Tokenizers Introduce Unfairness Between Languages</a></li><li><a href=\"https://aclanthology.org/2023.emnlp-main.614/\" rel=\"nofollow ugc noopener\">Ahia et al. (2023): Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models</a></li><li><a href=\"https://huggingface.co/LumiOpen/Viking-33B\" rel=\"nofollow ugc noopener\">Viking model card (LumiOpen)</a></li><li><a href=\"https://arxiv.org/abs/2207.04672\" rel=\"nofollow ugc noopener\">NLLB Team (2022): No Language Left Behind (FLORES-200)</a></li><li><a href=\"https://aclanthology.org/2022.sumeval-1.4/\" rel=\"nofollow ugc noopener\">Federmann et al. (2022): NTREX-128</a></li></ul>","headings":[{"level":1,"text":"The Nordic token tax","id":"the-nordic-token-tax"},{"level":2,"text":"Why does a non-English prompt cost more?","id":"why-does-a-non-english-prompt-cost-more"},{"level":2,"text":"What is a token?","id":"what-is-a-token"},{"level":2,"text":"What we measured","id":"what-we-measured"},{"level":2,"text":"Every model charges more","id":"every-model-charges-more"},{"level":2,"text":"It is not the language","id":"it-is-not-the-language"},{"level":2,"text":"A case for local AI labs","id":"a-case-for-local-ai-labs"},{"level":2,"text":"References","id":"references"}]}}