Benjamin BERGEN – 14 septembre 2023
Titre : LLMs are impressive but we still need grounding to explain human cognition
RÉSUMÉ :
Human cognitive capacities are often explained as resulting from grounded, embodied, or situated learning. But Large Language Models, which only learn on the basis of word co-occurrence statistics, now rival human performance in a variety of tasks that would seem to require these very capacities. This raises the question: is grounding still necessary to explain human cognition? I report on studies addressing three aspects of human cognition: Theory of Mind, Affordances, and Situation Models. In each case, we run both human and LLM participants on the same task and ask how much of the variance in human behavior is explained by the LLMs. As it turns out, in all cases, human behavior is not fully explained by the LLMs. This entails that, at least for now, we need grounding (or, more accurately, something that goes beyond statistical language learning) to explain these aspects of human cognition. I’ll conclude by asking but not answering a number of questions, like, How long will this remain the case? What are the right criteria for an LLM that serves as a proxy for human statistical language learning? and, How could one tell conclusively whether LLMs have human-like intelligence?
BIOGRAPHIE :
Ben BERGEN is Professor of Cognitive Science at UC San Diego, where he directs the Language and Cognition Lab. His research focuses on language processing and production with a special interest in meaning. He’s also the author of 'Louder than Words: The New Science of How the Mind Makes Meaning' and 'What the F: What Swearing Reveals about Our Language, Our Brains, and Ourselves.’
RÉFÉRENCES:
Trott, S., Jones, C., Chang, T., Michaelov, J., & Bergen, B. (2023). Do Large Language Models know what humans know? Cognitive Science 47(7): e13309.
Chang, T. & B. Bergen (2023). Language Model Behavior: A Comprehensive Survey. Computational Linguistics.
Michaelov, J., S. Coulson, & B. Bergen (2023). Can Peanuts Fall in Love with Distributional Semantics? Proceedings of the 45th Annual Meeting of the Cognitive Science Society. Austin, TX: Cognitive Science Society.
Jones, C., Chang, T., Coulson, S., Michaelov, J., Trott, T., & Bergen, B. (2022). Distributional Semantics Still Can't Account for Affordances. Proceedings of the 44th Annual Conference of the Cognitive Science Society. Austin, TX: Cognitive Science Society.