How to implement AI (Without burning a hole in the budget)
The cost controls I’d put around an AI rollout, before, during and after the pilot: attribution, alerts and hard caps through to prompt caching and showback.
Every department I talk to wants to “do something with AI”. The pattern is depressingly familiar. Someone senior reads that AI will deliver 40% efficiency gains, a prototype appears within a fortnight, access quietly spreads across the organisation, and three months later finance wants to know why there’s a five-figure Bedrock bill that nobody can explain or own.
The technology usually works fine. The problem is that nobody put any cost controls in before turning the taps on.
This post covers the controls I’d put in place, and when. My experience is with Claude on Amazon Bedrock in UK government, so the tooling I mention (AWS Budgets, QuickSight, application inference profiles) is AWS-flavoured. Azure OpenAI and Vertex AI have equivalents for all of it, and the principles hold whichever model you choose.
I’m also assuming assurance is done: governance board happy, data residency sorted, route to live agreed. If it isn’t, go and do that first.
Why AI spend catches people out
AI spend doesn’t behave like the rest of your cloud bill. You pay per token, so cost follows how people use the tool rather than anything you provisioned. An extra paragraph in a system prompt gets sent with every call. A retry loop at 2am can get through a month’s budget before anyone’s awake. A new team can start using it on Monday without finance knowing.
So the controls need to be in before anyone gets access. Bolting them on after the first big invoice is too late.
Agree the numbers, then start small
Before anything goes live, agree the money side with your finance business partner: the budget ceiling, the unit cost you’re aiming for (per user, per case, per summary), how they want spend broken down, and whether the department can cope with variable spend or needs a fixed monthly cap.
Tie every use case to an outcome you can measure, or cost per outcome means nothing. Pick something where the underlying process works, too. AI won’t fix a broken workflow, it’ll just make the mess faster.
Then roll out to a small group first. Five to ten engaged users on one use case will tell you what a hundred would cost, let you fix runaway prompts before they become a runaway bill, and give you a credible cost per outcome before anyone commits budget to scaling. Every department I’ve seen go straight from prototype to directorate-wide access has spent the next quarter firefighting spend they could have predicted from a two-week closed pilot.
The controls I’d insist on before the pilot
Attribution from day one. If everyone shares one Bedrock invocation path, nobody knows who’s spending what, and six months later you’re piecing usage together from CloudTrail. You need to slice spend by user, team, model, use case and cost centre so finance can map it to the P&L. In practice that means an application inference profile per use case, user and cost centre tags on every invocation, per-team IAM roles, and the model ID logged on every call. My test is simple: can you say how much the casework summariser cost last month, which team drove it and on which model, in under thirty seconds? If not, none of the other controls will work properly.
Tiered alerts. Alerts at 50%, 75% and 90% of the monthly budget, going to the product owner and the finance business partner. The aim is that nobody gets surprised, because surprises in government finance cost careers.
A hard cap. Alerts get ignored, so you also need something that actually stops the spend. An AWS Budget Action can remove the IAM permission to call Bedrock once the monthly cap is hit. Leave enough headroom for normal growth, but keep it tight enough that a runaway retry loop can’t burn through the annual budget over a weekend.
A dashboard a non-technical person can read. Spend by use case, team, user and model, tokens in and out, cost per successful interaction, and the trend over 30 days, 90 days and six months. The SRO should be able to open it in a steering committee and know within ten seconds whether things are on track. If it needs an analyst to translate it, you’ve built a data source.
Model selection discipline. At current list prices, Opus costs around five times as much per token as Haiku, and most use cases don’t need it. Default to Sonnet, drop to Haiku where accuracy allows, and make people justify Opus in writing. That one policy takes a noticeable chunk out of the bill.
Invocation logging. Turn on Bedrock model invocation logging to CloudWatch or S3, so you can see who called which model, from where, and how many tokens it used. It’s dull right up until the NAO or internal audit come asking, and then it’s the difference between a calm conversation and a very bad week.
Once the pilot is stable
When the basics are bedded in, the next layer is about catching problems earlier and getting more out of every pound.
- Cost anomaly detection scoped to Bedrock, to catch the spikes that mean something’s broken (a retry loop, a prompt-injection attack inflating tokens, a new team ramping up unannounced) rather than genuine demand.
- Prompt caching. Long system prompts get resent with every call, and caching can cut input costs on that repeated context by up to 90%.
- Batch inference for anything that isn’t urgent, at half the on-demand price.
- Unit economics, moving the conversation from total spend to cost per outcome: per case triaged, per summary written, per pull request reviewed. That’s the number the SRO cares about.
You can run a pilot without these. By month three of scaling, you’ll wish you had them.
Running it as a platform
Once several directorates are using it, cost control becomes an organisational question as much as a technical one.
- Showback or chargeback, so cost lands with the team driving it rather than a central IT budget.
- Rate limiting at the application layer, so one misbehaving client can’t drain the shared budget.
- Provisioned Throughput for predictable workloads where the commitment works out cheaper than on-demand.
- Automated prompt evaluation that flags when a prompt change pushes up the average token count, before it ships.
- FinOps-by-design gates, so every new use case gets a cost review as part of normal governance.
Costs don’t stand still
Three things will move your numbers after the pilot.
Your cost profile changes shape. Pilot spend is mostly experimentation: lots of prompt tweaking and test runs, few real users. Production spend is volume. The cost per outcome you measured with ten users won’t hold with a thousand, so re-baseline as you grow and be straight with finance about the curve.
Models change underneath you. Every new Claude release shifts the trade-off between price and performance. Re-test your prompts against new models with real cost and quality data. A team still on the model it picked eighteen months ago is probably overpaying, getting worse output, or both.
Prompts drift. Treat the system prompt as production code: version it, review it, and check the cost impact of every change. A one-line addition can push tokens per call up by 20%, and nobody notices until the invoice arrives.
Where to start
None of this is difficult. It’s product management and FinOps applied to a type of spend that behaves a bit differently from the rest of the cloud bill. Agree the numbers, put the controls in before you turn the taps on, start small, and keep measuring.
If you’re setting up something similar and want a hand getting it right, get in touch. Otherwise, take what’s useful and ignore the rest.
Tell me about your project
A few lines about the work and when you want to start are enough. I usually reply within one working day.
Available from December 2026