A token is the unit of text that language models read and write: a small piece of text, sometimes a whole word and sometimes only part of one.
In short
- A token is a piece of text the model processes as one unit. Everything sent to the model, and everything it writes, is counted in tokens.
- According to OpenAI, in English one token is about 4 characters or 0.75 words.
- A model has a length limit: the instructions, the input and the answer together can't exceed it.
- In OpenAI's API you pay by tokens, separately for input and for output.
What is a token, in one image
Think of Lego. You don't build a house from sand but from bricks, and bricks of different sizes. A language model doesn't read letters or whole sentences, but "text bricks" like these. A short, common word may be one brick, and a long or rare word may break into several.
Tokens are the unit in which the model counts its own work: how much it read, how much it wrote and how much room it has left.
How does it work?
OpenAI gives a rule of thumb for English text: a token is about 4 characters or 0.75 words. Note that this is a rule for English. In Hebrew and other languages the split may be different, so don't rely on it for an exact calculation. If you need a number, check it with the token-counting tool from the model provider.
Source: OpenAI documentation, Concepts and the API pricing page. The rule of thumb refers to English.
The documentation also defines the context window: the instructions and input sent to the model, together with the answer it generates, can't exceed the model's maximum length. Otherwise something doesn't fit.
Why does this matter for a business?
- Cost. OpenAI's pricing page shows prices per million tokens, with input and output priced separately. Many models also offer a discount on cached input. What you pay depends on how much text is sent and how much is generated. The components of an agent's cost are broken down in the article on the cost of an AI agent for business.
- Long documents. Summarizing a contract or a long catalog consumes many tokens, and can push everything else out of the window.
- Long conversations. The longer a conversation gets, the more history there is to send to the model, so you can start a new conversation when you change topic.
- Short, clear instructions. A concise instruction saves tokens.
- Agents. An agent that works in several steps reads and returns text at every step, so it's worth asking whoever sets it up to explain how usage is measured. On how an agent is built: What is an AI agent.
What we do in practice
A business owner doesn't need to count tokens by hand. What you should demand from anyone who sets up an agent for you: a clear explanation of how usage is measured, and how waste is prevented, for example by sending a whole document when a section of it is enough. The best way to understand costs is to check the provider's current rate, because prices change.
Common myths
Myth"A token is exactly a word"
In practiceOpenAI speaks of an approximation: about 0.75 words per token, in English. So a single word can be more than one token.
Myth"You only pay for the answer"
In practiceIn the API you pay for input and for output, at different rates.
Myth"You can paste as much text into the model as you like"
In practiceThere is a length limit. The input and the answer together can't exceed it.
Myth"The number of tokens is the same in every language"
In practiceOpenAI's rule of thumb is stated for English. In other languages the number may be different, so check in practice.
Related terms
- What is an LLM: the language model that processes tokens.
- What is RAG: a way to bring the model only the relevant passages, instead of a whole document.
- What is an AI agent: a system that works in steps and consumes tokens in each one.
- What is an AI hallucination: when the model produces an answer that sounds right and isn't.
- How much does an AI agent cost for a business: how tokens enter the cost.
Frequently asked questions
What is a token in AI?
A token is the unit of text that language models read and write. It can be a whole word, part of a word or a punctuation mark. The token count determines how much room the text takes in the model's length limit, and when using an API, also how much you pay.
How many tokens are in a word?
OpenAI says that in English one token is about 4 characters or 0.75 words. That is a rule of thumb for English only. In Hebrew the number may be different, so use a counting tool for an exact figure.
What is a context window?
It is the model's length limit. According to OpenAI's documentation, the instructions and input together with the generated answer can't exceed the model's maximum context length.
How do tokens affect cost?
In OpenAI's API, prices are shown per million tokens, and input and output are priced separately. The more text is sent and generated, the higher the cost. The exact amount depends on the model and on the current rates on the provider's site.
