Estimate LLM token costs before you roll out: a simple method with Python

#The goal

Someone asks, "What will this chatbot cost each month?" You can give a sensible estimate without guessing, using token counts, expected traffic and your provider's current price list.

This post doesn't quote prices, because they change often. Look up the current figures for your model and region.

#What you pay for

Most LLM APIs charge per token. A token is a small piece of text. As a rough guide, one English token is about four characters, or three-quarters of a word.

Hidden token use is the common surprise. The user may type 20 words, but your app might send 3,000 tokens of instructions and documents along with them.

#Step 1: count tokens for a sample prompt

pip install tiktoken

import tiktoken

enc = tiktoken.get_encoding("o200k_base")  # used by recent OpenAI models

system_prompt = "You are a helpful IT support assistant. Answer using only the context."
context = open("sample_context.txt", encoding="utf-8").read()
question = "How do I reset my VPN token?"

parts = {
    "system": len(enc.encode(system_prompt)),
    "context": len(enc.encode(context)),
    "question": len(enc.encode(question)),
}
print(parts, "total:", sum(parts.values()))

tiktoken works for OpenAI models. Other providers tokenize differently, so treat its count as an estimate for them, or use their own token-counting tool. Chat formatting adds a few more tokens per message.

#Step 2: measure real usage

Real API responses report what you'll be billed for. Prefer these figures to any estimate:

from openai import OpenAI

client = OpenAI()  # reads OPENAI_API_KEY from the environment

response = client.chat.completions.create(
    model="your-model-name",
    messages=[
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": f"{context}\n\nQuestion: {question}"},
    ],
    max_tokens=500,
)

usage = response.usage
print("input:", usage.prompt_tokens, "output:", usage.completion_tokens, "total:", usage.total_tokens)

Run 20-50 realistic questions, including long ones, and save the numbers. Use the average and the 95th percentile, rather than one example.

import statistics

inputs = [1850, 2300, 2100, 3900, 1700]    # from your test runs
outputs = [220, 310, 180, 480, 250]

print("avg in:", statistics.mean(inputs), "avg out:", statistics.mean(outputs))
print("max in:", max(inputs), "max out:", max(outputs))

#Step 3: build the estimate

The formula:

cost per request = (input tokens  / 1,000,000 x input price per million)
                 + (output tokens / 1,000,000 x output price per million)

monthly cost = cost per request x requests per day x days per month

As a script, with your own numbers:

INPUT_PRICE_PER_M = 0.0    # fill in from your provider's price page
OUTPUT_PRICE_PER_M = 0.0   # fill in from your provider's price page

def monthly_cost(users, requests_per_user_day, avg_in, avg_out, days=22):
    requests = users * requests_per_user_day * days
    cost = (requests * avg_in / 1_000_000 * INPUT_PRICE_PER_M
            + requests * avg_out / 1_000_000 * OUTPUT_PRICE_PER_M)
    return requests, round(cost, 2)

for name, users, per_day in [("low", 200, 2), ("expected", 500, 4), ("high", 2000, 8)]:
    print(name, monthly_cost(users, per_day, avg_in=2500, avg_out=300))

Always give a range, so nobody treats a guess as a promise. If a stakeholder wants one figure, give the expected case and state your assumptions.

#Step 4: add what people forget

A chat session adds up fast. With a 2,000-token base and 300 tokens added per turn, the input for the tenth turn is about 4,700 tokens rather than 2,000.

#Step 5: put limits in place

An estimate won't stop a runaway loop. Add limits before launch.

Limit the answer length in each request:

response = client.chat.completions.create(
    model="your-model-name",
    messages=messages,
    max_tokens=500,
)

Limit how much you send:

MAX_INPUT_TOKENS = 6000

def trim_history(messages, enc):
    while sum(len(enc.encode(m["content"])) for m in messages) > MAX_INPUT_TOKENS and len(messages) > 2:
        messages.pop(1)  # keep the system prompt, and drop the oldest turn
    return messages

Set a budget alert at your provider or cloud account, and record usage per feature or team. On Azure, that means Cost Management budgets. On AWS, see AWS Budgets.

Warning

Budget alerts are usually delayed, and they don't stop spending by themselves. For a hard stop, see cost kill switches.

Caution

Don't put real customer data or secrets in test prompts, unless your organization has approved that provider for it. Use sample data for estimates.

#Common mistakes

#Check your estimate after launch

Compare the forecast with the real numbers each week for the first month:

• Average input and output tokens per request
• Requests per day, and how much the heaviest users send
• Actual bill from your provider

Then update your model. If the bill is higher than expected, see controlling output tokens and prompt caching.

#Prevent it next time

• Log usage from each response, with a feature name, so that you can trace spend.
• Re-run your estimate whenever you change the model or the amount of retrieved context.
• Learn the basics in your first LLM API call.