AI responds to 35% of financial queries incorrectly

New research has sparked concerns about the usage of AI for answering finance questions, after findings revealed that over one third of AI-generated answers were incorrect.

The study, conducted by broker analysis site Investing in the Web, posed 100 finance and investment-related questions to ChatGPT, the most popular generative AI chatbot.

These questions varied between different categories in the world of finance, such as ‘How to save for my child’s education?’, ‘How does the average pension compare to the average salary?’, and ‘What are the pros and cons of investing in gold?”.

The study also asked both evergreen questions, where the right answer will always remain relatively consistent, and current questions, where the answers are more dependent on recent information.

The responses from ChatGPT were considered ‘correct’ by the industry experts in 65% of cases, which is a reasonably high result. However, in 29% of cases, the response was incomplete or misleading, and in 6% of cases, the answer was entirely wrong.

This means that over one third of the questions posed to the AI bot were answered incorrectly or unsuitably.

Pedro Braz, CEO of Investing in the Web, commented: “As with any other tool, ChatGPT serves its purpose if used in moderation. A good rule of thumb is to use it early in the research process, to help clarify terms or shed light on a general topic. Using it to ask more personalised questions is where the risk of miscommunication comes into play.

“It is always advisable to ask for and check the listed sources from ChatGPT’s answers, and to cross-reference its main points with other trusted sites, especially when dealing with financial inquiries that may have a significant impact on our lives.

“For information and data that is subject to change such as interest rates, the current performance of specific stocks, or the average pension, it is usually best to go to the very source of the information, rather than asking AI chatbots for data, as ChatGPT has well-recognised issues with up-to-date information.

“Overall, ChatGPT search is a welcome application that can save us a lot of time, but it can also misdirect us if we’re not careful while using it. Where the subject matter is as serious as personal finances, overuse of AI can have a significant negative impact.”