Researchers make groundbreaking discovery about ChatGPT after testing it with 2,400 year-old math problem
Researchers have made a groundbreaking discovery after testing OpenAI's ChatGPT with 2,400 year-old math problem.
While there's plenty to worry about when it comes to AI's impact on the world or how it might harm us humans (if it came down to it), perhaps figuring out how the technology actually 'thinks' could alleviate some of those fears.
Scientists in Cambridge may have uncovered how ChatGPT thinks after testing the artificial intelligence chatbot with a 2,400-year-old maths puzzle.
They wanted to find out if the AI would use knowledge it already ‘held’ or develop its own solutions.
Advert
In the study titled ‘An exploration into the nature of ChatGPT’s mathematical knowledge’, the team asked the OpenAI chatbot to solve the ‘doubling the square’ problem.
First described by ancient Greek philosopher Plato, the brain teaser involves teaching somebody without mathematical knowledge how to double the area of a square.
Many would incorrectly double each side length. However, Plato noted that with the right 'prompts,' he could help people understand that the new square’s sides should be the same length as the diagonal of the original.
To their surprise, the researchers revealed that ChatGPT didn't take its usual approach of regurgitating its pre-existing knowledge of the famous solution.
Instead, the AI appeared to tackle it like a 'learner' would.
“When we face a new problem, our instinct is often to try things out based on our past experience,” said Dr. Nadav Marco, a visiting scholar at the University of Cambridge. “In our experiment, ChatGPT seemed to do something similar. Like a learner or scholar, it appeared to come up with its own hypotheses and solutions.”
ChatGPT's first attempt used an algebraic method that was unheard of in Plato's time, which only resulted in an approximate solution.
Then, when the team told the chatbot they were disappointed it couldn't find a 'elegant and exact' answer, it soon turned around to deliver the geometrical answer.
“If it had only been recalling from memory, it would almost certainly have referenced the classical solution of building a new square on the original square’s diagonal straight away,” said Andreas Stylianides, a professor of Mathematics Education at Cambridge. “Instead, it seemed to take its own approach.”
The findings suggest that AI models like ChatGPT mix together stored information with 'on-the-fly reasoning' to solve problems.
And with the right prompts, the world's most popular chatbot can provide the correct answer as humans would.
“These are core skills we want students to master, but it means using prompts like, ‘I want us to explore this problem together,’ not, ‘Tell me the answer,’” Dr. Marco added.
The research was published in the International Journal of Mathematical Education in Science and Technology.