How Do Kids Outlearn AI?

For at least 100,000 years, human kids have been the only ones capable of picking up a language and becoming totally fluent - it's like they have a knack for it. But now, with the emergence of large language models like ChatGPT, we've got a new player in the game. Still, there's a huge disparity between the amount of data these models need to learn and what kids require - this is what's known as the data efficiency gap, and it's a real head-scratcher for cognitive scientists and AI researchers.

The amount of data needed to train these large language models is just staggering - we're talking over 15 trillion tokens for some of them. Meanwhile, kids can learn a language with a fraction of that, maybe around 100 million words or so by the time they're a preteen. To put that in perspective, if you were to print out all the data needed for a large language model, it'd be a stack of paper that'd dwarf the International Space Station - whereas the data a kid needs would barely reach 20 meters high. It's a pretty wild difference.

Figuring out how kids learn language so efficiently could be a game-changer for AI research. If scientists can reverse-engineer the process, they might be able to create more data-efficient AI models that could be super useful - think training AI on video or creating chatbots for minority language communities. Plus, testing hypotheses about human learning in machine models could help answer some of those enduring questions about language and how kids' minds develop - like, are we born with a language instinct, or can we learn it just from experience?

The thing is, we still don't fully understand how kids learn language - it's a complex process, and researchers are still exploring different theories. Noam Chomsky, a linguist, has suggested that babies are born with a kind of hardwired knowledge of grammar, but that's still up for debate. More research is needed to really get to the bottom of it. By studying how kids learn language, researchers hope to gain a deeper understanding of this phenomenon and develop more efficient AI models that can learn from less data - which would be a huge breakthrough.

Source: MIT Technology Review