Cost, caching & chargeback
Two questions finance actually asks: who spent this, and could it have been less. aigw answers both from the same request log that already backs the audit trail, so the numbers reconcile with each other by construction.
Chargeback
Spend is attributed per virtual key, and keys carry identity: an owner, a team, a project, a cost centre. Observe → FinOps slices the bill by any of those, so “AI cost” stops being one line on a card statement and becomes a number each team can see and answer for.
Every row separates gross from net: what the calls would have cost without optimisation, and what you were actually billed.
Savings that do not change your output
aigw only claims a saving where the response is byte-identical to the unoptimised call. That is a deliberate limit. Trimming context or silently downgrading a model would also cut the bill, and would make every number in this page an estimate rather than a fact.
Three techniques qualify:
- Prompt caching. A repeated prefix is marked so the provider bills it at the cache rate.
- Exact-match cache. An identical request inside your configured TTL is served from cache.
- Coalescing. Concurrent identical requests become one upstream call, fanned out to every caller.
Each is recorded per request with its baseline, so the saving is auditable rather than asserted.
Why cache hit rate is the wrong number
Agentic workloads are input-dominated. A coding agent resends a large, mostly unchanged prefix on every turn, so whether that prefix is being read from cache matters far more to your bill than which model you chose.
But a hit rate flatters you. Writing a prefix to cache costs more than not caching it; reading it back costs far less. So the number that decides whether caching is paying for itself is reads per written token, not hits per request.
With a typical 1.25x write and 0.1x read, break-even is around 0.28 reads per written token. Below that, caching is costing you money while the hit rate looks respectable. FinOps shows the ratio against that threshold, and names any model that paid the write premium and read nothing back - the case an average would hide.
Guardrails on spend
Budgets are enforced at the gateway, before a request reaches a provider, so a cap is a limit rather than a report. See virtual keys & budgets.