AI tokens are the small units of text, audio, or images that a language model reads and writes. Models count everything in tokens: how much they can read at once, how long they take to answer, and usually what they cost.
A token is often a short word, part of a longer word, or a punctuation mark. Google's Gemini documentation puts a token at about four characters, and 100 tokens at about 60 to 80 English words.
How text and audio become tokens
Text. A tokenizer splits text into pieces from a fixed vocabulary and gives each piece a number. Common words tend to be one token. Rare words, product codes, and account numbers often split into several.
Languages. The same sentence can take very different numbers of tokens in different languages. A NeurIPS 2023 paper by Aleksandar Petrov, Emanuele La Malfa, Philip H.S. Torr and Adel Bibi found differences of up to 15 times for translated text.
Models. Token counts also change from one model to the next. Anthropic's pricing page says the tokenizer in its newer models produces about 30% more tokens for the same text.
Audio and images. Models that take audio directly turn it into tokens as well. Google's documentation counts audio at 32 tokens per second and a small image at 258 tokens. In a voice pipeline that transcribes speech first, the model sees the transcript as text tokens.
Input tokens and output tokens
Input tokens are everything sent to the model in one request. That covers instructions, the conversation so far, retrieved documents, and results from tools such as an account lookup.
Output tokens are what the model writes back. For models that reason before answering, they include reasoning steps the customer never sees. Anthropic bills these thinking tokens as output tokens.
On each new turn, the input usually carries the whole conversation so far plus the new message. Anthropic's context window documentation describes this pattern. A long conversation therefore costs more per turn as it goes on.
How token counts drive cost and latency
Cost. Providers usually price input and output tokens separately, per million tokens. Anthropic's pricing page, read November 14, 2026, shows the pattern. Every model in its main table charges five times as much per output token as per input token. Google's documentation also ties cost to input and output token counts.
Caching. Text that repeats on every request, such as instructions and policy, can often be cached. Anthropic prices a cache hit at a tenth of the base input price on most models, per the same page.
Latency. A longer answer takes longer to generate. OpenAI's latency guide lists "generate fewer tokens" among its principles. It also notes that a long prompt can delay the first token, especially when much of the input is new. On a phone call, that delay is silence the caller hears (see turn-taking).
Tokens in a customer service conversation
The arithmetic below is illustrative, with round numbers and no prices.
A chat agent receives about 2,000 tokens of instructions and policy text on every turn. A 150-token account lookup stays in the conversation too. Each customer message is about 25 tokens, and each reply about 60.
Over 10 turns, the agent writes 600 output tokens. Every turn resends the instructions and the growing history, so input adds up to about 25,600 tokens. Input outnumbers output by roughly 40 to 1.
At a five-to-one price ratio, input would still be close to 90% of this conversation's model cost. Trimming fixed instructions, caching them, and keeping retrieved passages short therefore often save more than shorter replies.
A voice call adds its own load. If a model took a five-minute caller recording as audio at 32 tokens per second, that alone would be 9,600 input tokens.
Frequently asked questions
How many words is 1,000 tokens?
For English, about 600 to 800 words, using Google's rule of thumb of 60 to 80 words per 100 tokens. Other languages, numbers, and codes usually take more tokens per word. The provider's token counter gives the exact figure for a given model.
Do tokens limit how long a customer conversation can be?
Yes, indirectly. Each request has to fit in the model's context window, which is counted in tokens. Every turn resends the history, so a long call or chat grows toward that limit. Many agents summarize older turns and keep fixed instructions short, which keeps long conversations within the limit and cheaper per turn.
Why does the language of a conversation change the token count?
Tokenizers split some languages into more pieces than others, so the same message can take several times as many tokens after translation. Petrov and colleagues measured gaps of up to 15 times. For a contact center serving several European languages, it helps to measure token use per language before estimating volumes.
How can a customer service AI agent use fewer tokens?
Common steps are shorter fixed instructions, caching the parts that repeat, and retrieving only the policy sections a request needs. Summarizing older turns helps in long conversations. Tool results can be short too, returning the three fields the agent needs instead of a whole customer record.
How we approach this at Zowie
Zowie's AI agents work on voice, chat, and email, with over 100 million conversations a year across our clients. The language model speaks. A separate, deterministic Decision Engine returns every amount, rate, offer, disclosure, and escalation from your rules, with an audit log. To see it on your own conversation types, you are welcome to book a demo.
Explore: Large language model (LLM), Context window, Natural language processing (NLP), Multilingual support, AI observability & traces, Turn-taking