Subscribe
Logo
Logo
  • Topics Icon Topics
    • AI Icon AI
    • Banking Icon Banking
    • Blockchain/DeFi Icon Blockchain/DeFi
    • Embedded Finance Icon Embedded Finance
    • Fraud/Identity Icon Fraud/Identity
    • Investing Icon Investing
    • Lending Icon Lending
    • Payments Icon Payments
    • Regulation Icon Regulation
    • Startups Icon Startups
  • Podcasts Icon Podcasts
  • Products Icon Products
    • Webinars Icon Webinars
    • White Papers Icon White Papers
  • TechWire Icon TechWire
  • Search
  • Subscribe
Reading
Navigating the Hallucination: GenAI’s Imperfection in FIs
ShareTweet
human robot
Home
AI
Navigating the Hallucination: GenAI’s Imperfection in FIs

Navigating the Hallucination: GenAI’s Imperfection in FIs

Isabelle Castro Margaroli·
artificial intelligence
·Aug. 21, 2023·4 min read

“How is GenAI being used by financial services?” said Robert Antoniades. Co-Founder and General Partner of Information Venture Partners. “The simple answer is it’s not being used. Certainly not broadly.”

Given the amount of airtime GenAI has been getting in the conversations around innovations in financial services, this answer may be a surprise. However, Antoniades explained while AI is used all over the financial system, “The specific use cases of our interpretation of Gen AI, which is Chat GPT, I would struggle to find many use cases for that.”

Artificial Intelligence (AI) has been used increasingly in the financial services sector since the sixties. A recent report published by Bain Capital Ventures (BCV) stated that over 75% of companies of financial services companies now use AI for their operations. 

ai use in FI

Generative AI and Large Language Models (LLMs) are seen by many to have extensive potential, too, targeting areas where traditional AI has had its shortcomings.

“As sand can fill the open space in a jar of stones, generative AI can fill the gaps within financial services organizations left unfulfilled by traditional AI,” reads the BCV report. “These gaps today represent the unfulfilled potential of financial services organizations to offer better, higher quality, less expensive experiences for customers, more rewarding and fulfilling experiences for employees, and better earnings for shareholders.”

Throughout the fintech community, companies are announcing their exploration and application of GenAI. Areas that could benefit from the technology span financial services, from personalized customer service to improved underwriting. Where traditional AI has been applied, innovators are now striving to extract more, turning to the application of GenAI. But, while there is potential, many feel current applications fall short of the mark. 

“If ChatGPT is the iPhone, we’re seeing a lot of calculator apps,” said Christina Melas-Kyriazi, a partner at Bain Capital Ventures to the Financial Times earlier this year. “We’re looking for Uber.” 

A reason for this, in many cases, is its accuracy. 

Generative AI’s Imperfection

“The concern with Generative AI is its hallucinations or errors,” said Sam Bobley, CEO of Ocrulus.

The GenAI “hallucination” refers to instances when AI models produce answers or outcomes that are deemed “nonsensical” or “false”. While AI experts aren’t sure why exactly it occurs, possible explanations turn to the complexity of language and insufficient data. In the more publicized instances, lawyers have created arguments based on made-up precedents with non-existent citations, and journalists have published articles filled with fake news. 

These occurrences, while troublesome for the aforementioned lawyers and journalists, could be catastrophic if applied in finance. 

“If you’re applying for a small business loan or a mortgage, and the AI makes a mistake, and the wrong person is getting approved, or the lenders lose a ton of money, it’s a massive problem,” Bobley continued.

A particular area financial innovators have been exploring for GenAI application is financial advice. GenAI chatbots could be trained to become hyper-personalized financial advisors, able to communicate directly with consumers and tailoring responses to meet their specific needs. Consequently, this application could reduce barriers to accessing financial services and improve financial inclusion. 

RELATED: Why GenAI is More than Hype

However, due to possible hallucinations, applying pure GenAI to financial advisory services is still out of reach. “In financial services, if it’s anything important, it has to be 100% accurate,” said Antoniades.

“When you make a deposit in your bank account, you want to know the money’s there. It’s not that it can be there 99.9% of the time. It’s there. When they give you advice, they really should be 100% accurate, not 90% accurate…There’s no room for errors.”

Human in the Loop

While an answer to the hallucination issue could be restricting GenAI’s usage to areas that don’t need 100% accuracy, other solutions exist. Many are turning to a symbiosis of GenAI and human interaction. 

Antoniades explained that in many cases, GenAI could be used as a tool to generate an initial response that is then used by employees as a starting point for further exploration. “When you think about LLMs, they’re actually horizontal by nature. They scour all the information out there and then present it to you. And I think there’s a place for that,” he said. By doing so, GenAI can make workflows more efficient, allowing humans to only spend time focusing on specific areas and automating more manual tasks. 

Due to GenAI’s ability to evolve based on interaction, “feedback loops” can be created to help the tool reach 100% accuracy. Humans can check outcomes generated by the AI and feed in corrections that the AI can use for future answers. “Prompt engineering” is also a possible solution proposed by experts, allowing humans insight into the AI’s thought process and identifying issues. While still in need of human oversight, this process of troubleshooting, leading to increased accuracy, can improve the efficiency of automation.

“Because humans are focusing on fewer issues, and there’s automation in the process, you shorten the duration of which these processes take place. They’re thereby generating revenue for the institution much faster,” he continued. 

While a future where financial services are powered purely by AI is, in many cases, still only theoretical, the inclusion of it into a workflow including humans could still make its mark.

RELATED: The Fintech Coffee Break – Joe Robinson, Hummingbird

  • Isabelle Castro Margaroli
    Isabelle Castro Margaroli

    Isabelle is a journalist for Fintech Nexus News and leads the Fintech Coffee Break podcast.

    Isabelle's interest in fintech comes from a yearning to understand society's rapid digitalization and its potential, a topic she has often addressed during her academic pursuits and journalistic career.

    View all posts

Tags
AIfintechgenAI
Related

Ready to Sign Up for a Gen AI Certification Program? Fintech Founders & Others Weigh In on the Trend

Expanding Business Revenue Streams with a Fintech Podcast

Lawsuit, subpoena, have fintechs revisiting AI

Rising digital spending among findings of Alkami, Cornerstone Advisors report

Popular Posts

Today:

  • Revised-AI-InvoiceAI Faces Skepticism. Startups Say: OK, Pay When it Works Jun. 25, 2025
  • Ahead of AIOutsmart Pricing Objections Before They Arise with AI Jul. 1, 2025
  • Globe-money-symbolsOPINION: Why Brazil and India are leading the global digital shift through payment innovation Jun. 24, 2025
  • Paraform Founders, Jeffrey Li and John KimFunded: Paraform raises $20M to put top recruiters, not AI, in the driver’s seat Jun. 27, 2025
  • Stylizedhouse-with-EKGFintech x the One Big Beautiful Bill Jun. 26, 2025
  • TechNexus The AI IssueSteal Like an AI? Defining Fair Use & Creativity Jun. 25, 2025
  • PayabliFunded: Payments infrastructure co Payabli lands $28M Series B to AI-ify Jun. 20, 2025
  • DanMurphy-FN-headshotCFPB’s Next Open Banking Battle Begins Jun. 3, 2025
  • Current stablecoin adoptionWhy Banks (and Fintechs) Need to Embrace Stablecoins Today Jun. 12, 2025
  • GreenliteAI-Alex-WillGreenlite AI is on a mission to revolutionize banking compliance Jun. 10, 2025

This month:

  • WP UmbrellaTo Bank or Not to Bank: The ILC Question Jun. 5, 2025
  • DanMurphy-FN-headshotCFPB’s Next Open Banking Battle Begins Jun. 3, 2025
  • GreenliteAI-Alex-WillGreenlite AI is on a mission to revolutionize banking compliance Jun. 10, 2025
  • Current stablecoin adoptionWhy Banks (and Fintechs) Need to Embrace Stablecoins Today Jun. 12, 2025
  • ai-work-nexusWalkMe Vets Declare War on SaaS Bloat with $10M Seed for Autonomous Agents Jun. 10, 2025
  • Ben Hemani, Founding Partner at Bison VenturesThe Risk and Reward of Betting Big on AI’s Next Frontier Jun. 4, 2025
  • Jon StonaTips from Airwallex x McLaren on Making the Best of a Fintech Sponsorship  Jun. 18, 2025
  • Ironclad State of AI ReportThe Economics of AI Trust Jun. 11, 2025
  • Email-AI-pieceAvatar CEOs Have Entered the Meeting Jun. 18, 2025
  • TechNexus The AI IssueMeeker’s AI Bombshell + The VC Betting on AI Reshaping The Physical World  Jun. 4, 2025

  • About
  • Contact
  • Disclaimer
  • Privacy Policy
  • Terms
Subscribe
Copyright © 2025 Fintech Nexus
  • Topics
    • AI
    • Banking
    • Blockchain/DeFi
    • Embedded Finance
    • Fraud/Identity
    • Investing
    • Lending
    • Payments
    • Regulation
    • Startups
  • Podcasts
  • Products
    • Webinars
    • White Papers
  • TechWire
  • Contact Us
Start typing to see results or hit ESC to close
lis digital banking USA Lending Club UK
See all results