An AI Dream of NPV
I have dreamed a dream, but now that dream is gone from me.
~ Morpheus in the film The Matrix
I recently met a friend for coffee at Starbucks to catch up and talk tech. We worked together in finance at a major bank. Bank CEOs are proclaiming to responsibly use generative AI. My friend and I discussed different ways that AI could be used to solve complex accounting, finance, and regulatory problems. We started playing around with DALL-E and the subject of hallucination came up. I did some research before this discussion; and, I came prepared with some examples. I had a conversation with a different friend about how ChatGPT might “make up” web links and financial calculations. I thought I could take a peek into the topic of financial hallucinations using a simple financial calculation — net present value — but let’s define an AI hallucination before delving into the details.
Hallucinations
The Cleveland Clinic defines human hallucinations as false perceptions of sensory experiences. OpenAI, the firm that created ChatGPT and DALL-E, describes AI hallucination as a tendency to invent facts in moments of uncertainty. An AI hallucinations can be described as the phenomenon of a machine, such as a chatbot, generating seemingly realistic sensory experiences that do not correspond to any real-world input.
Permutations
In physics, there are 3 laws that can predict 99% of events. In finance, there are 99 laws that can predict 3% of events. In AI, just one question can produce a near infinite amount of answers depending on the phrasing of the prompt. A friend of mine was using ChatGPT for everything from job descriptions to real estate investment research. My friend let me know that he observed hallucinations when trying to crunch numbers on potential investments. I thought that I’d try to find a hallucination-generating AI prompt somewhere in the field of finance. Why not try calculating the net present value of a few cash flows at 10% interest? I’ll ask ChatGPT and maybe some other genAI programs and Simple enough — right?
Net Present Value
The problem: Calculate the net present value at 10% of the following annual cash flows: -1000, 12000, 1200, 1200, 1200, 1200. I’ve used ChatGPT to answer a bunch of questions from movies to coding. I have not previously submitted prompts requiring calculations where the accuracy could be scrutinized.
NPV = $3,651.70 according ChatGPT. I guess that sounds reasonable…but I should probably validate the responses in Excel using its trusty built-in finance functions!
The plain vanilla Excel NPV formula only takes in the interest rate and cash flow array as inputs. Excel-calcualted NPV = $3,226.31. Hang on a minute…something’s not right. A closer look at the ChatGPT outputs shows some miscalculations that we can call subtle hallucinations. The 2 NPV calculations are off but looking at annual calculated values (underlines in red) shows the errors.
Miscalculations
The miscalculations!? The errors!! The horror! I drilled down to the yearly components of the NPV calculation to see where the problem was. Year 1 was okay. After year 1, ChatGPT started approximating the calculations. The Year 5 calculation produced different answers (See image above ChatGPT NPV Highlighted Miscalculations).
Period!?
I showed the results to my accountant friend at Starbucks and he asked a critical question — were the cash flows at the the beginning or end of the period? Great point. Time to draw — sans DALL-E. Let’s start with a cash flow diagram. The diagram will visualize cash flows occurring at the beginning and the end of the period.
Recalculating
Excel has a lot of finance functions. The NPV() function does not have a specific input for beginning or end of period. However, the FV() or Future Value function does provide that flexibility. Take note that the first year for beginning of period calculations is 0 and 1 for end of period. Also, ‘Type’ in the FV() function is an argument for whether cash flows are made at the beginning (=1) or end of the period (=0). So, the recalculating will commence by first using the FV() function to calculate the future value and then discount to the present at the hurdle rate of 10% per annum.
Beginning of Period
Let’s retry the prompt and explicitly state that cash flows occur at the “beginning of the period”. Am I prompt engineering now? Hmmm. New Present Value equation…aaaaannnddd very weird output. $5,226.31. That can’t be right. Time to model the beginning of period calculation in Excel.
It’s time for the end…of period NPV calculation.
End of Period
Hmmm. End of Period NPV = $3,551.60. That’s closer to a NPV value validated in Excel…but that happens to be the Beginning of Period calculation.
The Other Guys
I also tried the same series of prompts in other gen AI programs including Bard AI from Google and Claude.ai. Bard got the calculation correct down to the penny for Beginning of Period. Beginning of Period NPV calculation was the default without specifying in the prompt. A key difference with Bard is that it used code (Python!) to calculate the NPV. I used Bard when it first launched and I was underwhelmed that it couldn’t do things like translate given Google’s vast capabilities. Bard has been advancing by leaps and bounds on a weekly basis. Google is incremently adding it’s vast technological capabilities to Bard. It’s exciting to see how it will continue to develop.
In Conclusion
This started off as whimsical curiosity and then expanded into an exercise in prompt engineering. Net Present Value is a very simple calculation compared to the complexity found in the modern world. A relatively simple example demonstrates how much the world will need subject matter experts that know the results of that the AI prompts should generate. The future will belong to the capable individuals with deep functional expertise and great skill attaining the desired response from AI machines. Don’t be afraid of the unknown. The water is warm. Jump in the sandbox and get started!
