AI Is Becoming a Bank Risk-Control Problem, Not Just a Tech Experiment

New Treasury and NIST frameworks show how quickly artificial intelligence is moving from innovation programs into the risk architecture of financial institutions.

For banks, the artificial intelligence conversation is beginning to sound less like a technology pitch and more like a risk committee meeting. What data can an AI system access? Who is responsible when an automated agent acts on behalf of an employee? Can a bank reconstruct what happened after an AI system makes a recommendation or initiates an action? And how should institutions test systems whose behavior can change depending on the data, model, and environment around them?

Those questions are becoming harder to postpone.

Nikhil Goel, a Founding Engineer and Engineering Manager at cloud data security and AI governance company Theom.ai, has spent the past several years working on security infrastructure used in major U.S. financial institutions. He has watched what was once largely a theoretical security problem move rapidly into production environments.“Two years ago, ‘how do you secure an LLM’ was a theoretical question,” Goel said. “Today it is something every enterprise I work with is actively trying to solve.”

The policy landscape is moving in the same direction.

In February, the U.S. Department of the Treasury released a Financial Services AI Risk Management Framework designed specifically for financial institutions. Two months later, the National Institute of Standards and Technology began developing a separate AI Risk Management Framework profile for critical infrastructure, including sectors where failures can have consequences far beyond a single company.

Now NIST is also seeking public input on a draft framework for testing, evaluating, verifying, and validating AI systems, with comments open through October 6.

Taken together, the initiatives point to a change in how policymakers and financial institutions are thinking about artificial intelligence. The first phase of enterprise AI was largely about capability: what can the models do?

The next phase is increasingly about proof: can an organization demonstrate that the system behaved as intended?

That distinction matters particularly in financial services.

Banks already operate under layers of operational, cybersecurity, privacy, model-risk, and compliance controls. Artificial intelligence does not replace those obligations. In many cases, it makes them more complicated.

An AI assistant might summarize information from several internal systems. An agent might eventually interact with those systems on behalf of an employee. A model might help flag suspicious transactions or support a credit, fraud, or compliance workflow. Each use case creates value.

Each also raises questions about data access, identity, explainability, and accountability.

Treasury's new financial-services framework is an acknowledgment that broad principles alone are no longer enough. It adapts NIST's existing AI Risk Management Framework to the operational and regulatory realities of financial institutions, with an emphasis on accountability, transparency, resilience, and risk management throughout the AI lifecycle.

The government is not telling banks to stop using AI. Quite the opposite. The frameworks are increasingly aimed at making deployment possible without forcing institutions to choose between innovation and control.

That is where engineers working inside large financial environments are seeing the problem evolve in real time.

Traditional enterprise security was built largely around people and applications.

A person logs into a system. The organization knows who that person is. Permissions determine what he or she can access. Logs record activity.

AI agents complicate that model. That concern is already moving into practice, with six banks recently publishing voluntary agentic commerce principles focused on trust, security, and accountability as AI agents begin to influence transactions.

If an agent is operating for an employee, should it inherit every permission that employee has? If it retrieves information from several systems and combines it into a new output, do the restrictions attached to the original data still follow that output? If the agent takes an action automatically, which identity should appear in the audit trail?

“An agent shouldn't get a blanket copy of an employee's permissions,” Goel said. “It should carry that person's identity on every request, so the access decision and the audit record both point back to an accountable human.”

These are not philosophical questions for a bank. They are architecture questions.

Goel's work has focused in part on identity-aware controls for AI systems and on data lineage, the ability to trace where information came from, how it changed, and where it traveled.

Those same problems are reflected in two patent applications on which, according to company materials, Goel is the lead inventor. One concerns identity-aware security for LLMs and AI agents; the other focuses on tracking data lineage across transformations in systems that do not naturally expose how information has changed.

Both concepts are becoming more important as financial institutions move beyond simple chatbot use cases.

A customer record may start in one regulated system, move into a cloud warehouse, be transformed by another application, and eventually become part of the context supplied to an AI model. An institution may still need to know what restrictions apply to that information even after several transformations.

Without reliable lineage, that can become difficult.

The same is true for identity. An AI system that acts for an employee may operate much faster and across far more information than the employee would manually. That makes preserving the boundaries around that person's authority more important, not less. The broader policy direction increasingly reflects those concerns.

NIST's developing critical-infrastructure profile is intended to help operators translate broad AI trustworthiness principles into requirements that developers, vendors, and internal teams can actually use. Its concept note points to areas such as guardrails for autonomous cybersecurity agents, traceable and auditable AI recommendations, human oversight, and systems that remain reliable when operating conditions change.

NIST's newer TEVV-Athlon draft pushes the conversation further toward testing.

TEVV stands for testing, evaluation, verification, and validation. The terminology may sound procedural, but the underlying question is straightforward: how does an organization know that an AI system is actually fit for the environment in which it will operate?

That is a familiar problem in finance.

Banks do not generally deploy critical infrastructure based on a product demonstration alone. Systems have to survive real transaction volumes, real failure conditions, and real regulatory scrutiny.

AI governance is beginning to face the same standard.

Goel has seen that gap between a working system and a production-ready system firsthand. During an earlier deployment at a large financial institution, Theom encountered data volumes far beyond those of its previous customers. Queries that had worked at a smaller scale began timing out.

According to company data, the underlying processing engine had originally handled roughly 100,000 queries per day. As larger institutions came onto the platform, the system eventually had to support more than 500 million daily queries while bringing average response times down from roughly two seconds to about 200 milliseconds.

Goel led the redesign of the query path, data partitioning, and distributed processing architecture to handle the increase.

The episode was fundamentally a scaling problem, but it also illustrates something increasingly relevant to AI risk management: controls that work in a demonstration are not necessarily controls that work inside a bank.

A governance system that can identify sensitive information in a small environment may behave differently across billions of data interactions. An access-control rule that makes sense on paper may fail if identity context disappears as information moves between systems. An AI evaluation that performs well during testing may not capture how the system behaves once the surrounding data or operating environment changes.

This is one reason the financial sector may end up becoming an important proving ground for enterprise AI governance.

Banks already have mature risk functions and a long history of evaluating third-party technology, models, cyber threats, and operational resilience. In many cases, AI risk will have to be incorporated into those existing disciplines rather than managed by a standalone AI team.

Treasury's framework explicitly takes that approach. It is designed to be flexible enough for institutions of different sizes and levels of AI maturity, while creating a more consistent vocabulary for technical, legal, compliance, and business teams.

That common language may become increasingly important. One of the difficulties with AI governance today is that the same system can look very different depending on who is examining it.

A data scientist may be focused on model performance.

A security engineer may care about access boundaries. A compliance team may want evidence that restricted information was handled correctly. A business executive may care primarily about whether the system improves productivity. All of those questions can be valid at the same time.

The challenge is creating controls that connect them.

There is also a competitive dimension.

Financial institutions do not want risk frameworks that make it impossible to deploy AI while faster-moving competitors capture the productivity gains. Treasury has explicitly positioned its new resources as tools intended to reduce uncertainty and support wider AI adoption, not simply restrict it.

That creates a delicate balance.

Too little governance can expose institutions to security, compliance, and operational risk. Too much friction can leave useful technology trapped indefinitely in pilot programs.

The likely winners will be institutions that can turn governance into infrastructure rather than bureaucracy. That means knowing where sensitive data is, preserving its context as it moves, constraining what automated systems can do, and generating enough evidence to understand what happened afterward.

It also means testing AI systems against the environment they will actually encounter rather than assuming a successful demonstration is sufficient.

None of the new federal frameworks provide a complete answer.

NIST's critical-infrastructure profile is still being developed. Its TEVV-Athlon framework remains in draft form. Treasury's financial-services framework is designed as a practical risk-management resource rather than a substitute for existing regulation.

But the direction is becoming increasingly clear.

As AI becomes embedded in the systems that move money, detect fraud, analyze risk, and handle sensitive customer information, governance will no longer sit on the edge of the technology conversation.

It will become part of the operating architecture.

For financial institutions, the next important AI question may therefore be less about what the technology can do.

It may be whether the institution can prove that it remained in control while the technology was doing it.