From Bias to Systemic Challenges, Treasury Secretary Janet Yellen Flags AI Risks in Finance

Treasury Secretary Janet Yellen believes the use of artificial intelligence (AI) in finance carries significant risks even as it benefits financial firms. Yellen spoke at an AI conference held on Thursday at the US Treasury Department and on Friday at the Brookings Institution, where she said that AI "offers tremendous opportunities for the financial system" while outlining that AI-related risks will be a major concern in the coming years.

"For many years, the predictive capabilities of AI have supported forecasting and portfolio management," Yellen said. "AI's ability to detect anomalies has contributed to efforts to combat fraud and illicit finance. Many customer support services have been automated. Across these and many other use cases, we've seen that AI, when used appropriately, can improve efficiency, accuracy, and access to financial products," she added.

"Specific vulnerabilities may arise from the complexity and opacity of AI models; inadequate risk management frameworks to account for AI risks; and interconnections that emerge as many market participants rely on the same data and models," she continued.

After this, she talked about how too many participants rely on the same AI models as well as the same set of data, which could push the existing biases or create new ones that impact decision-making in financial markets.

She then discussed the potential of AI which can reduce transaction costs. However, she also raised concerns saying, "Concentration among vendors developing models, providing data, and providing cloud services may also introduce risks, which could amplify existing third-party provider risks. Insufficient or faulty data could also perpetuate or introduce new biases in financial decision-making." 

She did acknowledge how "advances in natural language processing, image recognition, and generative AI" can create new opportunities and also make many financial services affordable and easier to access. 

"Scenario analysis, often used by firms and governments to understand opportunities and risks in the context of uncertainty, could also be beneficial," she said. "Given how quickly AI technology is developing, with fast-evolving potential use cases for financial firms and market participants, scenario analysis could help regulators and firms identify potential future vulnerabilities and inform what we can do to enhance resilience."

The main challenges of AI in finance

While AI is known for making a lot of human work smooth, the adoption of AI in fianace does pose a lot of threats. For starters, the finance sector adhers to regulations that were not made for AI. Ensuring AI systems also work by these regulations can be a major challenge in this sector. Other challenges include data privacy as AI systems require access to large volumes of sensitive data.

Other threats include cybersecurity as well as algorithmic bias. This can happen where AI systems perpetuate existing biases that can lead to unfair discriminatory financial practices. Other than this, AI which largely depends on interconnectivity can also lead to systemic risks within the sector. This is why, the experts believe that managing AI in the finance sector where even one small mistake can lead to disaster is extremely risky.