Andrew Lampinen

First in our unplugged series live from #NeurIPS2022 We discuss natural language understanding, symbol meaning and grounding and Chomsky with Dr. Andrew Lampinen from DeepMind. We recorded a LOT of material from NeurIPS, keep an eye out for the uploads. YT version: https://youtu.be/46A-BcBbMnA References [Paul Cisek] Beyond the computer metaphor: Behaviour as interaction https://philpapers.org/rec/CISBTC Linguistic Competence (Chomsky reference) https://en.wikipedia.org/wiki/Linguistic_competence [Andrew Lampinen] Can language models handle recursively nested grammatical structures? A case study on comparing models and humans https://arxiv.org/abs/2210.15303 [Fodor et al] Connectionism and Cognitive Architecture: A Critical Analysis https://ruccs.rutgers.edu/images/personal-zenon-pylyshyn/proseminars/Proseminar13/ConnectionistArchitecture.pdf [Melanie Mitchell et al] The Debate Over Understanding in AI's Large Language Models https://arxiv.org/abs/2210.13966 [Gary Marcus] GPT-3, Bloviator: OpenAI’s language generator has no idea what it’s talking about https://www.technologyreview.com/2020/08/22/1007539/gpt3-openai-language-generator-artificial-intelligence-ai-opinion/ [Bender et al] On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? https://dl.acm.org/doi/10.1145/3442188.3445922 [Adam Santoro, Andrew Lampinen et al] Symbolic Behaviour in Artificial Intelligence https://arxiv.org/abs/2102.03406 [Ishita Dasgupta, Lampinen et al] Language models show human-like content effects on reasoning https://arxiv.org/abs/2207.07051 REACT - Synergizing Reasoning and Acting in Language Models https://arxiv.org/pdf/2210.03629.pdf https://ai.googleblog.com/2022/11/react-synergizing-reasoning-and-acting.html [Fabian Paischer] HELM - History Compression via Language Models in Reinforcement Learning https://ml-jku.github.io/blog/2022/helm/ https://arxiv.org/abs/2205.12258 [Laura Ruis] Large language models are not zero-shot communicators https://arxiv.org/pdf/2210.14986.pdf [Kumar] Using natural language and program abstractions to instill human inductive biases in machines https://arxiv.org/pdf/2205.11558.pdf Juho Kim https://juhokim.com/

Andrew Lampinen

Senior Research Scientist

DeepMind

Please note that in this interview Dr. Lampinen was expressing his personal opinions and they do not necessarily represent those of DeepMind. Patreon: https://www.patreon.com/mlst Discord: https://discord.gg/ESrGqhf5CB YT version: https://youtu.be/yPMtSXXn4OY Dr. Andrew Lampinen is a Senior Research Scientist at DeepMind, and he thinks that symbols are subjective in the relativistic sense. Dr. Lampinen completed his PhD in Cognitive Psychology at Stanford University. His background is in mathematics, physics, and machine learning. Andrew has said that his research interests are in cognitive flexibililty and generalization, and how these abilities are enabled by factors like language, memory, and embodiment. Andrew with his coauthors has just released a paper called symbolic behaviour in artificial intelligence. Andrew lead in the paper by saying the human ability to use symbols has yet to be replicated in machines. He thinks that one of the key areas to bridge the gap here is considering how symbol meaning is established, and he strongly believes it is the symbol users themselves who agree upon the symbol meaning, And that the use of symbols entails behaviours which coalesce agreements about their meaning. Which in plain English means that symbols are defined by behaviours rather than their content. [00:00:00] Intro to Andrew and Symbolic Behaviour paper [00:07:01] Semantics underpins the unreasonable effectiveness of symbols [00:12:56] The Depth of Subjectivity [00:21:03] Walid Saba - universal cognitive templates [00:27:47] Insufficiently Darwinian [00:30:52] Discovered vs invented [00:34:19] Does language have primacy [00:35:59] Research directions [00:39:43] Comparison to BenG OpenCog and human compatible AI [00:42:53] Aligning AI with our culture [00:47:55] Do we need to model the worst aspects of human behaviour? [00:50:57] Fairness [00:54:24] Memorisatation on LLMs [01:00:38] Wason selection task [01:03:45] Would an Andrew hashtable robot be intelligent? Dr. Andrew Lampinen https://lampinen.github.io/ https://twitter.com/AndrewLampinen Symbolic Behaviour in Artificial Intelligence https://arxiv.org/abs/2102.03406 Imitating Interactive Intelligence https://arxiv.org/abs/2012.05672 https://www.deepmind.com/publications/imitating-interactive-intelligence Impact of Pretraining Term Frequencies on Few-Shot Reasoning [Yasaman Razeghi] https://arxiv.org/abs/2202.07206 Big bench dataset https://github.com/google/BIG-bench Teaching Autoregressive Language Models Complex Tasks By Demonstration [Recchia] https://arxiv.org/pdf/2109.02102.pdf Wason selection task https://en.wikipedia.org/wiki/Wason_selection_task Gary Lupyan https://psych.wisc.edu/staff/lupyan-gary/

Andrew Lampinen

Senior Research Scientist

DeepMind