← Back to Blog

LLM Usage Reconciliation: Cache and Reasoning Token Pitfalls

Resolve LLM billing discrepancies by learning how OpenAI, Claude, and Gemini report input caching, write TTLs, and reasoning tokens without double counting.

Why LLM Usage Telemetry Fails Financial Audits

Engineering teams managing large language model workloads frequently discover that invoice totals exceed internal telemetry calculations. One possible cause is that internal accounting pipelines often treat reported token fields as independent metrics and aggregate them naively.

A metric labeled input_tokens or prompt_tokens does not represent identical mathematical sets across providers. Depending on whether you call native OpenAI endpoints, Claude Messages, or Gemini GenerateContent, cache reads, cache writes, and internal reasoning steps may represent subsets of base counters, additive counters, or separate operational tiers. Without normalizing payloads into mutually exclusive buckets prior to applying billing formulas, cost engines routinely double count cached inputs and reasoning tokens.

To establish an auditable ledger, your data platform must capture raw usage payloads alongside an explicit combination of provider endpoint and schema version before attempting normalization.

Cross-Protocol Field Semantics and Inclusions

Accounting models must reflect the official payload structures published by provider documentation rather than assuming uniform OpenAI-compatible conventions.

OpenAI Responses API Object

In the OpenAI Responses specification (OpenAI Responses Reference), usage.input_tokens reflects the total input volume. Inside usage.input_tokens_details, both cached_tokens (cache reads) and cache_write_tokens (cache writes) are subsets of input_tokens. If an internal accounting system bills input_tokens at standard rates and subsequently adds cache read or write tokens as extra lines, those inputs are billed twice.

Similarly, usage.output_tokens represents the complete output count. The field usage.output_tokens_details.reasoning_tokens denotes thought or chain-of-thought generation that is already included in output_tokens. Adding reasoning_tokens to output_tokens artificially inflates generation volume.

Anthropic Claude Messages API

Claude handles prompt caching with an additive schema (Claude Prompt Caching). The official total prompt consumption equals cache_read_input_tokens + cache_creation_input_tokens + input_tokens. Here, input_tokens represents solely the uncached portion of the prompt. If you subtract cache hits from input_tokens under Claude native semantics, you corrupt the base calculation.

Furthermore, cache write operations may carry distinct TTL intervals (such as 5-minute versus 1-hour retention windows). These require tier-specific rates rather than a single uniform write multiplier.

Google Gemini GenerateContent API

Google Gemini structures usage telemetry via usageMetadata (Gemini GenerateContent API). In this schema, promptTokenCount encompasses cachedContentTokenCount. Output is tracked under candidatesTokenCount, while internal generation steps appear under thoughtsTokenCount. The documented totalTokenCount reflects prompts, tool-use prompts, thoughts, and candidate outputs combined. Treating totalTokenCount as an output metric or treating cachedContentTokenCount as an additive cost item produces inaccurate ledgers.

Normalizing Telemetry into Mutually Exclusive Buckets

To prevent discrepancies, discard field-level summation across protocols. For the text-only OpenAI Responses example below, use four mutually exclusive buckets. Other protocols, TTLs and tool charges need their own mapping:

  1. Base Input Tokens: Uncached prompt tokens billed at full standard input rates.
  2. Cache Read Tokens: Tokens retrieved from active memory, billed at discounted read rates.
  3. Cache Write Tokens: Tokens written into the cache, categorized by their retention TTL.
  4. Total Output Tokens: Final completed generation, which already incorporates reasoning tokens under standard pricing models.

For an OpenAI Responses payload, the base input calculation is:

$$\text{Base Input} = \text{input_tokens} - \text{cached_tokens} - \text{cache_write_tokens}$$

The output volume remains strictly output_tokens. Reasoning tokens provide visibility into operational consumption patterns, but they must not be appended to the total.

Synthetic Reconciliation Walkthrough

To illustrate this calculation, consider a synthetic request under OpenAI Responses semantics. This example is strictly a mathematical demonstration rather than a measured provider invoice or an observed online run.

Assume a synthetic payload contains:

  • input_tokens: 10,000
  • cached_tokens: 6,000
  • cache_write_tokens: 2,000 (all allocated to a 30-minute TTL)
  • output_tokens: 1,000
  • reasoning_tokens: 400 (contained within output_tokens)

Evaluating the mutually exclusive buckets:

  • Base Input: $10,000 - 6,000 - 2,000 = 2,000$
  • Cache Read: $6,000$
  • Cache Write (30m): $2,000$
  • Output: $1,000$ (not $1,400$)

We benchmark this against the catalog entry for gpt-6.2-sol listed on the APIBox catalog endpoint (https://api.apibox.cc/api/pricing). For standard context lengths up to 272,000 tokens, the baseline billing expression corresponds to:

  • Base Input ($p$): $2.00 per million tokens
  • Cache Read ($cr$): $0.10 per million tokens
  • Cache Write 30m ($cc$): $2.50 per million tokens
  • Output ($c$): $10.00 per million tokens

Calculating baseline expenditure:

$$\text{Cost} = \frac{2,000 \times 2.00 + 6,000 \times 0.10 + 2,000 \times 2.50 + 1,000 \times 10.00}{1,000,000} = $0.0196$$

Applying the gpt-vip-1 group pricing ratio of 0.10 yields a offline estimate of $0.00196. Had the pipeline incorrectly calculated output as $1,000 + 400 = 1,400$ and treated input_tokens as purely base input while adding cache operations on top, the baseline calculation would have climbed to $0.0396 in the internal estimate.

Handling Telemetry Gaps and Idempotency

In distributed architectures, requests fail mid-stream or drop HTTP bodies due to network timeouts. When usage objects are absent, cost systems should not record zero. Mark the usage state as unknown.

Maintain two parallel accounting figures for internal cost allocation: known estimated subtotals and pending reconciliation counts. When requests are retried via automated agent workflows, record every physical request attempt separately and reconcile its actual billability, keyed by explicit correlation IDs. For guidance on handling distributed failures safely, read our analysis on /posts/agent-tool-timeout-idempotency/.

To audit real-world usage against provider invoices, inspect published rate charts on /pricing/ and validate baseline parameters via our catalog endpoint:

Base URL: https://api.apibox.cc/v1
Catalog API: https://api.apibox.cc/api/pricing

The pricing API confirms current catalog listings, tier thresholds, and enabled rate ratios. It does not validate real-time upstream capacity, caching eligibility, or protocol pass-through behavior. Register an account at APIBox Dashboard to test reconciliation pipelines against small, controlled batches before deploying automated financial ledgers to production.

This concerns internal estimates, not evidence of provider overcharging. Rates were checked on 2026-11-19. The calculator covers only standard-tier text input/output, excluding tools, storage, multimodal, Batch and other TTLs. Claude cache writes require separate 5-minute and 1-hour buckets; do not apply this calculator to Gemini. Restrict raw-usage access, exclude keys and prompt bodies, and retain billing period, timezone, attempt ID, price version and invoice line IDs.

Runnable offline check

Synthetic data only; no model calls or production retries.

from decimal import Decimal as D

# Fixed text-only catalog snapshot; not a generic billing adapter.
def estimate(base, read, write_30m, output):
    values = (base, read, write_30m, output)
    if any(type(v) is not int or v < 0 for v in values):
        raise ValueError("unknown or invalid token bucket")
    if base + read + write_30m > 272000:
        raise ValueError("outside this example's standard tier")
    rates = (D("2"), D("0.1"), D("2.5"), D("10"))
    return sum(D(v) * r for v, r in zip(values, rates)) / D("1000000") * D("0.1")

assert estimate(2000, 6000, 2000, 1000) == D("0.00196")
for bad in (True, -1, None, 1.5):
    try:
        estimate(bad, 6000, 2000, 1000)
    except ValueError:
        pass
    else:
        raise AssertionError("invalid input accepted")
print(estimate(2000, 6000, 2000, 1000))

Sources and live catalog

FAQ

Are reasoning tokens billed on top of output tokens in OpenAI Responses?

No. In the OpenAI Responses API specification, reasoning_tokens is a child field of output_tokens_details and is already included inside the output_tokens count. Adding it again double-counts the internal cost estimate; it does not establish a provider overcharge.

How do Claude cache read tokens differ from OpenAI cached tokens?

Anthropic Claude uses an additive model where total input equals cache_read_input_tokens plus cache_creation_input_tokens plus input_tokens. In contrast, OpenAI Responses treats cached_tokens as an internal subset of input_tokens.

How should missing usage payloads be handled during streaming network drops?

Record the usage state as unknown rather than zero. Track confirmed costs separately from pending unquantified requests to ensure internal ledgers avoid false precision.

Try it now, sign up and start using 30+ models with one API key

Sign up free →