Understanding token consumption

The brain powering a Mate is a large language model (LLM), the same technology behind every AI assistant. An LLM reads text and writes text, and counts it in tokens: small pieces of words. As a rough guide, 1,000 tokens are about 750 English words.

Input: the context the LLM works from

Every LLM works from the context it is given at each turn: the Mate's instructions, the conversation so far, your files and the results of the tools it calls. This context is the input. It grows as the conversation goes on, which is why input is by far the largest part of the consumption, and why caching matters.

Optimized by Allmates: less context sent to the LLM

When context optimization is enabled on the chat and a saving is reported, Optimized by Allmates appears as the first sub-row under Input, in both the message popover and the chat's Consumption section. Its negative value shows the input tokens that Allmates context optimization removed. These savings are approximate, even when the count comes directly from the optimizer.

Next to the actual Input value, a struck-through value prefixed with ~ shows the approximate input without optimization. It combines the actual input and the reported saving. When no saving is reported, neither the sub-row nor the struck-through value appears.

Context optimization reduces the context sent to the LLM. Provider caching reuses part of the input that is sent: Cached and New remain parts of the actual Input value. The optimization saving is separate from those parts and is not another segment of the bar or an estimate of money saved. It does not calculate your bill.

Cached: context the LLM reuses

Most LLM providers keep recently processed context in a cache. When the same context comes back on the next turn, it is read from the cache instead of being processed again. Cached input is processed at a fraction of the price of new input.

• Cached is the part of the input read from the cache.
• New is the part processed for the first time on this turn.
• Cache write is context stored in the cache so the next turns can reuse it. The cache only lasts a few minutes, and its duration depends on the model provider: after a long pause, the context has to be written again.

Models without a cache show 0 cached.

Output: the answer, reasoning included

Output is what the LLM writes: the answer you read, plus, on models that think before they answer, their reasoning. Reasoning is part of the output, never added on top of it.

A record, not a live meter

The figures are recorded once the message is complete. They describe that turn and never tell you whether the cache is still active.

The cost figure

Next to the title of the message popover and of the Consumption section, an approximate cost prefixed with ≈ can appear. It is in US dollars: the model’s catalog rate applied to the token counters of the message, or of the whole chat. It is an approximation, neither your organization’s invoice nor its credits. An amount under one cent reads < $0.01.

Only organization owners and admins see it.

It is absent when:

• the model runs on your organization’s own provider key;

• the pricing of a call could not be established;

• the message predates the feature;

• the chat total has not been recomputed yet: it updates with the next message.

Where to find them in mates

• On a message (desktop): click the small gauge next to the time of a Mate's answer. A popover shows the input split into cached and new, and the output.

• For the whole chat: open Chat info, then Details, then the Consumption section. It adds up every message, with Cache write and Reasoning when reported. This is also the access on mobile.

Token figures are not a bill: cached and new tokens are not priced the same way, and the cost figure stays an approximation.