First in a series on what it takes to run AI agents in production inside a financial institution. Later pieces go one layer deeper on the parts explored here: the evaluation layer, our agent builder, the agent harness, integrations, and the work of moving a legacy process onto agents and keeping it there.
The real build-versus-buy decision for AI agents in banking
A modern agent can research an entity, reason across conflicting evidence, use tools, and carry a multi-step investigation from intake to disposition. In a back office that has always run on analyst hours and human judgment, that type of impact will already transform your organization.
It is also the part of the problem that is already solved.
The foundation models work great and will keep getting better. So why do most agent projects in banking still fail? And fail in the same, predictable way? For those who have tried, it becomes quickly apparent that there’s a vast distance between a demo that impresses a room and a system an institution can rely on, at production volume, inside its own controls. That distance is where the build-versus-buy conundrum is almost always underestimated.
A demo is not the same as a system
In our experience working with numerous banking customers, the failure modes are consistent enough to name.
Institutional knowledge is harder to encode than it looks. An agent performs only to the standard it has been given, and in a bank that standard does not live in one clean document. It is spread across policies, SOPs, desk procedures, escalation rules, QA criteria, risk appetite, and years of prior decisions. Handing an agent your often messy documents does not transfer your organizational wisdom. Someone still has to decide which evidence is authoritative, where a deterministic rule applies and where judgment is required, what counts as sufficient investigation, and when a case is actually done. An SOP that says "check the corporate registry, then cross-check the website" leaves open which registry, what counts as a match, what to extract, and what to do when the sources disagree. Encoding that does not require better prompt-writing. It is reconstructing the operating logic behind the work, and that reconstruction is most of the build.
Integration is the critical path, not a final step. An agent creates value when it completes the workflow, not when it can reason about it. Completing the workflow means reaching case management, transaction data, KYC and KYB providers, screening platforms, registries, document stores, and internal systems, several of which will have no usable API. The agent can be finished long before it can touch the systems where the work lives. An agent that cannot reach the work has not reduced it.
Multi-step workflows compound failure. A single investigation may chain identity resolution, counterparty research, screening, ownership analysis, document review, policy application, and disposition. Each step can look strong on its own while the end-to-end case stays unreliable. Nine independent steps at 95 percent each succeed together only about 63 percent of the time. Good architecture narrows that gap. It does not close it. Task accuracy is not the same as case accuracy (what examiners actually care about), and so the case is the only measure that matters in production. And that measurement is much more difficult to make than appears.
Production requires a second system: evaluation. Getting an agent right once proves very little. Models change, tools change, prompts change, providers change, policies change, and every change is a chance to regress. Knowing whether the agent still performs means building the layer around it: ground truth, rubrics, scenario suites, regression tests, traces, versioning, and monitoring. Without it, every material change becomes another manual validation project. The team is not just building an agent. It is building the system that tells them whether the agent still works.
Banks raise the bar
These problems exist in every industry. Banking makes them even hairier because the cost of a wrong output is different in kind.
An agent here can touch onboarding, investigations, risk decisions, money movement, and sensitive customer data, all of it inside a control environment of compliance, model risk, internal audit, information security, and privacy. A bad answer does not just cost the bank monetarily. It becomes a missed risk, an incorrect disposition, an audit finding, or an even more costly remediation program with a regulator watching.
The data is harder too. Names cross alphabets and transliterations. Different people share identities. Companies sprawl across subsidiaries, affiliates, directors, and beneficial owners in jurisdictions that disagree with each other. Addresses are partial. Registries conflict. A missing result is not the same as a clean one. These are not edge cases to handle later. They are the work.
A general agent framework gives you models, tool calling, memory, and orchestration. It does not give you the financial-services intelligence to resolve those identities, trace that ownership, reconcile that evidence, or measure any of it against a real investigative standard. That still has to be built.
The question people ask, and the one they should
Framed honestly, the decision is not "can our team build an agent." For many institutions the answer is yes. The better question is: once it is built, what have we signed up to own?
The LLM spend, the line most build cases model with care, is the least concerning number in the budget. An internal build also means owning the agent harness, the financial-services skills, the integration layer, the evaluation corpus, the release process, and the subject-matter input that defines acceptable performance. None of it ends at launch. Models change, providers change, policies change, integrations break, and every new production case surfaces a new way to fail. An internal build is not a project with a finish line. It is a standing obligation.
What happens when the people leave
One of our senior leaders lived this before joining Bretton AI. At a global bank, a strong team led by his engineering expertise built a sophisticated set of internal tools to automate operational work. The tools were good. They shipped, they ran, and for a while they did exactly what they were built to do.
Then the team changed. People moved to other groups, others left the bank, and the specific tribal knowledge of how the tools were wired, why a threshold sat where it sat, what to do when a data source shifted, walked out with them. The tools degraded subtly, the way undocumented systems do, until they were producing work no one fully trusted and no one still on staff knew how to fix. The automation that was supposed to remove operational load became the operational load, and the work fell back onto people.
The main takeaway of course, is not that the team was careless. They were capable, and the tools worked the day they were released. The lesson is that an agent is not a deliverable you ship and walk away from. It needs continuous care and feeding: supervision, retraining as the work drifts, and someone who understands it well enough to fix it when the ground moves. The hardest part of production is not the launch. It is the years after, and those years depend on people staying, staying current, and staying interested. That is a fragile thing to build a control function on.
When building is still the right call
There is a feasible way to build, and it is still worth considering.
If a team decides the back office itself is part of their strategic moat, an area it wants to own, automate, and drive operating expense out of as a durable advantage, building can make sense. So can a genuine constraint, where security, architecture, or governance requirements rule out any outside platform. And so can the case where the agent platform is meant to become part of the institution's own differentiated product.
In each of those, the institution is choosing to permanently staff the problem: a dedicated team accountable for orchestration, integrations, evaluation, model changes, and the failures that follow. A strong engineering organization can carry that. The real question is whether rebuilding and maintaining that infrastructure is the highest-value thing that group could be doing, and whether the institution is prepared to keep staffing it long after the launch that justified it.
For most institutions, the honest answer is that the workflow is strategic and the infrastructure underneath it is not.
Why buying wins for most
Buying is the stronger path when you want to own the decisions without owning the machine that runs them. Two advantages decide it.
Time to value. A purpose-built platform puts a production-ready agent into the workflow in the time it would otherwise take just to assemble the surrounding stack. The first real case runs in days, not after months of building the agent harness, integrations, and evaluation from a blank page.
Quality from the start. You inherit a baseline shaped by accumulated financial-services work rather than discovering the hard cases one production incident at a time. The difficult populations, the reconciliation logic, the evaluation rubrics, and the integration patterns are already there.
This is what the Bretton AI Platform is built to give an institution: the production layer you would otherwise have to build and then keep alive.
- Builder turns your policies, SOPs, risk logic, and expected outputs into agents that reflect your standard. It reuses proven back-office skills where they fit and keeps your thresholds, escalation logic, and decision rules explicit.
- The agent harness manages the long-running, stateful reality of an investigation: the required checks, the evidence, the entity identity, the retries and failures across a case. The model handles judgment. The system enforces the parts that should never depend on a model remembering them.
- Integrations connect the agent to where the work actually happens, across case management, data providers, registries, warehouses, files, and legacy applications, so cases flow in and results flow back to the system of record.
- Trust Infrastructure reviews completed work against your standard, surfaces material discrepancies and recurring failure patterns, and feeds regression testing, so a change to a skill, a model, or a tool can be tested before it reaches production.
You still define what matters: your policies, risk appetite, thresholds, and operating standard. You just do not have to build and forever maintain the system underneath them before any of it produces value.
The bottom line
Build-versus-buy is not a question about whether your engineers can call a frontier model. It is a question about how much has to exist around that model before an institution can trust it with real work, and about who keeps that system alive once the people who built it move on.
Build, and you start with model capability and spend months constructing the production system around it, then own that system indefinitely. Buy, and you start with most of that system already in place and shape it to your workflow. The point of the second path is not that you could not build. It is that you can grow the operation without building and staffing a new platform organization to hold it up.
What comes next in this series
This piece is the overview. The ones that follow expand it, one layer at a time:
- what an agent in production actually looks like and requires
- moving a legacy workflow onto agents without losing the standard that governed it
- Builder, and configuring an agent to an institution's own operating logic
- the agent harness that keeps long, stateful cases reliable
- integrations, and reaching the systems that have no clean way in
- evaluation, and knowing an agent still works after everything around it has changed
- what it actually takes to keep an agent healthy over years, not weeks
