LLM Token & Cloud GPU Cost Estimator
API cost
$13,997 a month
- Requests
- 2.59 million
- Input tokens
- 2.07 billion ($6,221)
- Output tokens
- 518.40 million ($7,776)
- Per 1,000 requests
- $5.40
Blended price $5.40 per million tokens. Prompt caching and batch discounts, where your provider offers them, can cut the input share substantially.
Self-hosted on rented GPUs
$4,320 a month
4 GPUs to keep up with 2,000 tokens/s, rented for the 360 active hours (100% busy). That is $9,677 a month less than the API.
With perfect autoscaling (paying only for busy GPU time) the floor is $4,320 for 1,440 GPU-hours. Real deployments need headroom for traffic peaks, and GPUs left running outside active hours cost extra. Engineering time, storage and networking are not included.
Throughput depends heavily on model size, quantization, batch size and context length. Input tokens are processed far faster than output tokens are generated, so a single tokens-per-second figure is a simplification; benchmark your own setup.
Estimate the monthly cost of an LLM workload from your traffic: requests and tokens per month, the API bill with input and output tokens priced separately per million, and what it would cost to serve the same load on rented GPUs.
Formula
Worked Example
Two requests a second for 360 active hours is 2.59 million requests. Input costs $6,221 and output $7,776, $13,997 in all. Serving 2,000 tokens a second takes four GPUs at 500 tokens/s each; renting them for the 360 active hours costs $4,320, but only because they would be 100% busy, so a real deployment needs extra capacity for peaks.
Frequently Asked Questions
Why are input and output tokens priced differently?
Each output token needs its own pass through the model, while input tokens are processed in parallel, so providers usually charge three to five times more for output. Long prompts with short answers are dominated by the input price; long answers by the output price.
How do I find tokens per second for a GPU?
Benchmark your model on the serving stack you plan to use at a realistic batch size and context length, and count input plus output tokens processed per second. Throughput varies more than tenfold between small and large models and with quantization, so published figures are only a starting point.
Why is self-hosting not always cheaper than the API?
The GPU estimate assumes the GPUs stay busy throughout the active hours. Real traffic is uneven, so you need headroom for peaks, and self-hosting adds engineering time, monitoring and storage. It tends to pay off only at steady, high volume with an open model that matches the API model's quality.