LLM Vendor Cost Comparison – API vs Self-Hosted
Monthly cost, cheapest first
2.07 billion tokens a month (1.30 B in, 0.78 B out) · 800 tokens/s while active
| Option | Per month | Per 1M tokens |
|---|---|---|
| Small API model$0.15 in / $0.60 out per 1M | $661 | $0.32 |
| Mid-size API model$0.80 in / $4 out per 1M | $4,147 | $2.00 |
| Self-hosted open model3 GPUs × 720 h × $3.50/h · 67% busy while active | $7,560 | $3.65 |
| Large API model$3 in / $15 out per 1M | $15,552 | $7.50 |
Compares serving cost only. Compare against API models of similar quality; a self-hosted model also needs engineering time, monitoring, storage and spare capacity for traffic peaks, none of which is included. API prompt caching and batch discounts are not modelled either.
Compare the monthly cost of one LLM workload across up to three API models, each with its own input and output price per million tokens, and a self-hosted open model on rented GPUs, billed either for the active hours only or around the clock.
Formula
Worked Example
The workload is 2.07 billion tokens a month (1.30 billion in, 0.78 billion out), or 800 tokens a second. Two GPUs could keep up, so three leave headroom at 67% busy; renting them all month costs $7,560, about half the large API model but nearly twice the mid-size one. Self-hosting only pays when it replaces an API model of similar quality.
Frequently Asked Questions
When does self-hosting an open model beat an API?
When traffic is high and steady enough to keep rented GPUs busy, and the open model is as good for your task as the API model it replaces. At low or spiky volume, paying per token is usually cheaper because idle GPUs still cost money.
Should GPUs be billed only for the active hours?
Only if you really shut them down. On-demand instances can be stopped outside busy hours; reserved instances and your own hardware are paid for around the clock, so choose 24/7 for those.
What does this comparison leave out?
Engineering and on-call time, monitoring, storage, networking and spare capacity for peaks on the self-hosted side; prompt caching and batch discounts on the API side. It compares price only, not quality, latency or reliability.