AI Tokens Explained: Why Your Bill Ignores Word Count

The invoice that made no sense

The first month I put an AI feature into one of my tools, I estimated the cost by word count. I knew roughly how many words users would run through it, multiplied by the per-token price, and budgeted accordingly. The bill came in meaningfully above my estimate, and my first assumption was a bug in my code. The bug was in my mental model: I had quietly treated one word as one token, and that is simply not how any of this is billed.

Tokens are the metering unit for every large language model, and they do not map cleanly onto words, characters, or anything else humans count naturally. Once I understood what a token actually is, the bills became predictable. This post is that understanding, in plain English, with the napkin math I now use before shipping anything.

What is a token, actually?

A token is a chunk of characters that the model treats as a single unit. Sometimes a token is a whole word, sometimes part of a word, sometimes a punctuation mark, sometimes a space glued onto the front of a word. The model never sees your text as letters or words. It sees a sequence of these chunks, and both your bill and every context limit are measured in them.

For ordinary English prose, OpenAI's own rule of thumb is that one token is roughly 4 characters, or about three-quarters of a word, which means 100 tokens is about 75 words. That ratio is the single most useful number in this post. But it only holds for typical English, and the interesting failures happen everywhere else, which is where the surprise bills come from.

The fastest way to make this concrete is to paste your own writing into the OpenAI tokenizer linked below, or into the AI token counter, and watch how the text splits. The word unbelievable is not one unit, and neither is your product name.

BPE in plain English

The splitting is done by an algorithm called byte pair encoding, and the idea is less exotic than the name. Start with individual characters. Scan an enormous pile of training text, find the pair of neighboring chunks that appears most often, and merge that pair into a new single chunk. Repeat tens of thousands of times. Frequent sequences like the, ing, and tion get promoted into single tokens, while rare sequences stay broken into pieces.

The consequence is a compression scheme tuned to what the model saw during training. Common English words cost one token. Rare words, invented words, and typos get split into fragments and cost several. A misspelled word can cost triple what the correct spelling costs, which delights me: it is the only billing system I know that charges extra for typos.

Why do code and non-English text cost more?

Code tokenizes expensively because of its texture. Indentation, brackets, underscores, and camelCase identifiers all fragment into extra tokens, and a variable named getUserAccountBalance splits into several pieces every time it appears. A screen of code regularly costs more tokens than a screen of prose, so features that process code deserve a bigger budget line from day one.

Non-English text can be much more expensive, because tokenizer vocabularies are trained mostly on English-heavy data. Languages in other scripts often need several tokens per word, and my own tests bear this out: the same short sentence translated into Japanese or Hindi can come back at a multiple of the English token count. Try one sentence in three languages in the tokenizer and you will see it immediately. If your product serves a global audience, the same feature quietly costs different amounts per language, which is worth knowing before the invoice teaches you.

Context windows: the other place tokens bite

The context window is the model's working memory, measured in tokens, and everything competes for it: your system prompt, the user's input, any documents you stuff in, and the space reserved for the reply. It is also where costs compound in a way that surprised me. In a chat interface, each new turn typically resends the whole conversation so far, so turn twenty carries nineteen turns of history as input tokens. Long conversations get more expensive per message as they go.

The standard defense is summarization: condense the history or the source document and feed the model the condensed version. I use the text summarizer for exactly this when preparing long material, and the token savings routinely dwarf every other optimization I have tried. Sending a model 30 pages when 2 pages of summary would do is the token equivalent of shipping a sofa to post a letter.

The napkin math I actually use

For planning, I start from words because words are what I can count in advance, usually with the word counter. Then: English prose, multiply words by 1.33 to estimate tokens. Code or mixed content, multiply by 2 as a deliberately pessimistic rule of thumb. Then check a real sample in the AI token counter, because a five-minute reality check beats a clever formula.

Worked example with invented but realistic pricing: imagine a model charging $3 per million input tokens and $15 per million output tokens. A 1,500-word article is roughly 2,000 tokens, so reading it costs about $0.006. Generating a 500-word reply, roughly 670 tokens, costs about $0.01. Two things jump out of that arithmetic. Single requests are almost comically cheap, so scale is the only thing that makes them expensive. And output tokens usually cost several times more than input, so an over-chatty model burns budget faster than a long prompt does. Asking for concise answers is a cost optimization, not just a style preference.

Where my estimates still go wrong

Even with the right ratios, a few line items kept surprising me until I started counting them deliberately. These four cover essentially every gap I have found between my estimate and a real bill.

Tokens also explain some model behavior that looks like stupidity but is really plumbing: models are famously bad at counting the letters in a word or reversing a string, because they never see letters, only chunks. Understanding the metering unit turns out to explain the failures as well as the invoice. On the writing side, the same instinct for compression pairs nicely with readability work, which I covered in what Flesch-Kincaid actually measures: shorter, plainer sentences are cheaper in tokens and easier on humans, a rare free lunch.

• The system prompt rides along with every single request, so 500 tokens of instructions is 500 tokens per call, forever
• Chat history gets resent every turn, so conversation length grows the per-message cost
• Formatting overhead is real: JSON keys, markdown tables, and HTML tags are all tokens
• Retries and validation failures bill twice, which my error handling generously demonstrated for me one weekend