Analyzing the GPT-3 AI’s Ability to Predict the Answer to Algebraical Questions

Abstract

AI Algorithms have been getting increasingly smart and complex since their conception. The Most Modern AI’s currently available are able to solve incredibly complex problems and are even able to solve abstract problems. It is only a matter of time until AI is able to supersede humans in Mathematical Calculations. (MGI et al. 2022). In this paper we took the most advanced AI currently available for commercial usage, GPT-3, and programmed it to use randomly generated mathematical input in order to test its solving abilities. By feeding this input, we evaluated whether the AIs ability to solve problems is solely based on complexity. We, however, found out that AI was also influenced by the type of operation. The distance and similarity metrics of many different AI Generated answers were taken and then compared with the actual ones.We conducted a series of T-tests to determine which results are statistically significant. We found out that the type of arithmetical expression in the problem, which was either addition, multiplication, division or subtraction, was significant (p=.0000027), suggesting that the type of problem matters more than previously expected.