Your cloud bill after the AI feature ships

Why this bill behaves differently

A normal web feature costs about the same whether it's used a lot or a little, because servers are billed by the hour. An AI feature often costs per request, based on how much text goes in and comes out. A popular feature, a long document or a loop that retries can all push the bill up quickly.

That's not a reason to avoid AI. It's a reason to measure before you launch.

What drives the cost

• Input size. Long prompts, big documents and large chat histories.
• Output size. Long answers cost more than short ones.
• Model choice. Larger models cost much more per request than smaller ones, often without a visible quality gain on simple tasks.
• Retries and loops. An agent that tries again, or calls other tools, multiplies the cost.
• Idle waste. Calling the model for requests where a lookup or rule would do.

Ways to keep it under control

• Use the smallest model that passes your tests. Run your evaluation set on a cheaper model and compare.
• Cache repeated answers. Many requests are near-duplicates.
• Trim the prompt. Send what the model needs, not everything you have.
• Set limits. Maximum length, maximum retries, maximum cost per task and per customer.
• Route simple things away. Don't use a model to do what a rule can.

What to do this month

1. Find your cost per request. Average and worst case, by feature.
2. Estimate cost per customer per month. Compare it to what they pay you.
3. Set a budget alert. At 50% and 80% of the monthly target, to a human.
4. Test a smaller model. On your evaluation set. Keep it if the results hold.
5. Add a hard cap per task. So a runaway loop can't produce a surprise invoice.