Artificial intelligence (AI) chatbots like Bard and ChatGPT are capable of producing humanlike text, but researchers question whether these models truly understand what they are saying. A paper by Emily Bender suggests that large language models (LLMs) used in chatbots generate text without reference to meaning, making them “stochastic parrots.” However, new research by Sanjeev Arora and Anirudh Goyal proposes that as LLMs get bigger and are trained on more data, they develop new abilities that hint at understanding combinations unlikely to exist in the training data. This theoretical approach has convinced experts like Geoff Hinton. Arora and Goyal used random graph theory to model LLM behavior and found that larger models become more skilled and gain unexpected abilities by combining multiple skills. They argue that these models are not simply mimicking what they’ve seen before. Testing their claim, they found that LLMs can generate text using multiple skills.
