Yoshua BENGIO – 17 février 2022
Titre : Conscious processing, inductive biases and generalization in deep learning
Résumé:
Humans are very good at “out-of-distribution” generalization (compared to current AI systems). It would be useful to determine the inductive biases they exploit and translate them into machine-language architectures, training frameworks and experiments. I will discuss several of these hypothesized inductive biases. Many exploit notions in causality and connect abstractions in representation learning (perception and interpretation) with reinforcement learning (abstract actions). Systematic generalizations may arise from efficient factorization of knowledge into recomposable pieces. This is partly related to symbolic AI (aas seen in the errors and limitations of reasoning in humans, as well as in our ability to learn to do this at scale, with distributed representations and efficient search). Sparsity of the causal graph and locality of interventions -- observable in the structure of sentences -- may reduce the computational complexity of both inference (including planning) and learning. This may be why evolution incorporated this as "consciousness.” I will also suggest some open research questions to stimulate further research and collaborations.
Bio:
Yoshua Bengio est professeur titulaire à l’Université de Montréal, fondateur et directeur scientifique de Mila – Institut québécois d’IA, et codirige le programme Apprentissage automatique, apprentissage biologique de CIFAR en tant que Senior Fellow. Il occupe également la fonction de directeur scientifique d’IVADO