GPT-4 can execute financial analysis better than humans

According to a recent study, GPT-4 may do better than human analysts when it comes to making short-term financial forecasts using statement analysis. The study, which was published in the Social Science Research Network (SSRN), discovered that in one- to six-month forecasts, GPT-4 achieved 60.31% accuracy compared to 56.7% for humans. The study does not, however, imply that AI models can take the position of human analysts.

The paper titled “Financial Statement Analysis with Large Language Models” attempted to find out the role conventional artificial intelligence (AI) models can play in analysing the financial statements of an organisation and predicting its performance in the stock market in the near future.

Such analysis has always been understood to be very complicated as a wide range of factors can influence the performance of companies. Even as some financial firms use artificial neural networks (ANN) to assist individuals in their analysis, large language models (LLMs) have not been used for this. The researchers wanted to see if a state-of-the-art (SOTA) LLM such as GPT-4 can be a valuable addition to this or not.

Researchers fed GPT-4 anonymised and standardised corporate financial statements (to prevent biases emerging from mentioning the company's name). Next, the researchers used two methods to test the capabilities of the LLM. The first was designing a simple prompt that directed the chatbot to analyse the statements and predict future earnings. The second was to use a “chain-of-thought' (CoT) prompt that taught the AI model to mimic financial analysts.

The CoT method asked GPT-4 to identify notable trends, compute key financial ratios, and to form expectations about future earnings. While the simple prompt did not fetch noteworthy results, the CoT prompts achieved 60.31 percent which was higher than the average human analyst's performance.

“We find that an LLM excels in a quantitative task that requires intuition and human-like reasoning. The ability to perform tasks across domains points towards the emergence of Artificial General Intelligence,” the paper stated.