Medical research made understandable with AI
Sorcero uses a mix of natural language processing, generative AI, and even more old school symbolic AI, where they craft their own ontologies, to try and ingest that river of new medical data and make it easier to search and comprehend.
Less than 0.2% of the global population can read a medical paper! AI can help make these dense works up to 700x more readable.
Medical Affairs Teams are the groups inside big pharmaceutical companies that helps surface the right information to health providers. It’s hard for them to keep up with the thousands of new articles and research papers being published each month, much less unpack that information.
Connect with Dipanwita Das and Hellmut Adolphs on LinkedIn.
Congrats to Lifeboat badge winner John Carrell for saving the question Self join vs. inner join with an excellent answer.
See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.
#88 Dr. WALID SABA - Why machines will never rule the world [UNPLUGGED]
Support us! https://www.patreon.com/mlst
Dr. Walid Saba recently reviewed the book Machines Will Never Rule The World, which argues that strong AI is impossible. He acknowledges the complexity of modeling mental processes and language, as well as interactive dialogues, and questions the authors' use of "never." Despite his skepticism, he is impressed with recent developments in large language models, though he questions the extent of their success.
We then discussed the successes of cognitive science. Walid believes that something has been achieved which many cognitive scientists would never accept, namely the ability to learn from data empirically. Keith agrees that this is a huge step, but notes that there is still much work to be done to get to the "other 5%" of accuracy. They both agree that the current models are too brittle and require much more data and parameters to get to the desired level of accuracy.
Walid then expresses admiration for deep learning systems' ability to learn non-trivial aspects of language from ingesting text only. He argues that this is an "existential proof" of language competency and that it would be impossible for a group of luminaries such as Montague, Marvin Minsky, John McCarthy, and a thousand other bright engineers to replicate the same level of competency as we have now with LLMs. He then discusses the problem of semantics and pragmatics, as well as symbol grounding, and expresses skepticism about grounded meaning and embodiment. He believes that artificial intelligence should be used to solve real-world problems which require human intelligence but not believe that robots should be built to understand love or other subjective feelings.
We discussed the unique properties of natural human language. Walid believes that the core unique property is the ability to do abductive reasoning, which is the process of reasoning to the best explanation or understanding. Keith adds that there are two types of abduction - one for generating hypotheses and one for justifying them. In both cases, abductive reasoning involves choosing from a set of plausible possibilities.
Finally, we discussed the book "Machines Will Never Rule The World" and its argument that the current mathematics and technology is not enough to model complex systems. Walid agrees with the book's argument but is still optimistic that a new mathematics can be discovered. Keith suggests the possibility of an AGI discovering the mathematics to create itself. They also discussed how the book could serve as a reminder to temper the hype surrounding AI and to focus on exploration, creativity, and daring ideas. Walid ended by stressing the importance of science, noting that engineers should play within the Venn diagrams drawn by scientists, rather than trying to hack their way through it.
Transcript: https://share.descript.com/view/BFQb5iaegJC
Discord: https://discord.gg/aNPkGUQtc5
YT: https://youtu.be/IMnWAuoucjo
TOC:
[00:00:00] Intro
[00:06:52] Walid's change of heart on DL/LLMs and on the skeptics like Gary Marcus
[00:22:52] Symbol Grounding
[00:32:26] On Montague
[00:40:41] On Abduction
[00:50:54] Language of thought
[00:56:08] Why machines will never rule the world book review
[01:20:06] Engineers should play in the scientists Venn Diagram!
Panel;
Dr. Tim Scarfe
Dr. Keith Duggar
Mark Mcguill
#68 DR. WALID SABA 2.0 - Natural Language Understanding [UNPLUGGED]
Patreon: https://www.patreon.com/mlst
Discord: https://discord.gg/HNnAwSduud
YT version: https://youtu.be/pMtk-iUaEuQ
Dr. Walid Saba is an old-school polymath. He has a background in cognitive psychology, linguistics, philosophy, computer science and logic and he’s is now a Senior Scientist at Sorcero.
Walid is perhaps the most outspoken critic of BERTOLOGY, which is to say trying to solve the problem of natural language understanding with application of large statistical language models. Walid thinks this approach is cursed to failure because it’s analogous to memorising infinity with a large hashtable. Walid thinks that the various appeals to infinity by some deep learning researchers are risible.
[00:00:00] MLST Housekeeping
[00:08:03] Dr. Walid Saba Intro
[00:11:56] AI Cannot Ignore Symbolic Logic, and Here’s Why
[00:23:39] Main show - Proposition: Statistical learning doesn't work
[01:04:44] Discovering a sorting algorithm bottom-up is hard
[01:17:36] The axioms of nature (universal cognitive templates)
[01:31:06] MLPs are locality sensitive hashing tables
References;
The Missing Text Phenomenon, Again: the case of Compound Nominals
https://ontologik.medium.com/the-missing-text-phenomenon-again-the-case-of-compound-nominals-abb6ece3e205
A Spline Theory of Deep Networks
https://proceedings.mlr.press/v80/balestriero18b/balestriero18b.pdf
The Defeat of the Winograd Schema Challenge
https://arxiv.org/pdf/2201.02387.pdf
Impact of Pretraining Term Frequencies on Few-Shot Reasoning
https://twitter.com/yasaman_razeghi/status/1495112604854882304?s=21
https://arxiv.org/abs/2202.07206
AI Cannot Ignore Symbolic Logic, and Here’s Why
https://medium.com/ontologik/ai-cannot-ignore-symbolic-logic-and-heres-why-1f896713525b
Learnability can be undecidable
http://gtts.ehu.es/German/Docencia/1819/AC/extras/s42256-018-0002-3.pdf
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
https://arxiv.org/pdf/2112.11446.pdf
DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learning
https://arxiv.org/abs/2006.08381
On the Measure of Intelligence [Chollet]
https://arxiv.org/abs/1911.01547
A Formal Theory of Commonsense Psychology: How People Think People Think
https://www.amazon.co.uk/Formal-Theory-Commonsense-Psychology-People/dp/1107151007
Continuum hypothesis
https://en.wikipedia.org/wiki/Continuum_hypothesis
Gödel numbering + completness theorems
https://en.wikipedia.org/wiki/G%C3%B6del_numbering
https://en.wikipedia.org/wiki/G%C3%B6del%27s_incompleteness_theorems
Concepts: Where Cognitive Science Went Wrong [Jerry A. Fodor]
https://oxford.universitypressscholarship.com/view/10.1093/0198236360.001.0001/acprof-9780198236368
#56 - Dr. Walid Saba, Gadi Singer, Prof. J. Mark Bishop (Panel discussion)
It has been over three decades since the statistical revolution overtook AI by a storm and over two decades since deep learning (DL) helped usher the latest resurgence of artificial intelligence (AI). However, the disappointing progress in conversational agents, NLU, and self-driving cars, has made it clear that progress has not lived up to the promise of these empirical and data-driven methods. DARPA has suggested that it is time for a third wave in AI, one that would be characterized by hybrid models – models that combine knowledge-based approaches with data-driven machine learning techniques.
Joining us on this panel discussion is polymath and linguist Walid Saba - Co-founder ONTOLOGIK.AI, Gadi Singer - VP & Director, Cognitive Computing Research, Intel Labs and J. Mark Bishop - Professor of Cognitive Computing (Emeritus), Goldsmiths, University of London and Scientific Adviser to FACT360.
Moderated by Dr. Keith Duggar and Dr. Tim Scarfe
https://www.linkedin.com/in/gadi-singer/
https://www.linkedin.com/in/walidsaba/
https://www.linkedin.com/in/profjmarkbishop/
#machinelearning #artificialintelligence
NLP is not NLU and GPT-3 - Walid Saba
#machinelearning
This week Dr. Tim Scarfe, Dr. Keith Duggar and Yannic Kilcher speak with veteran NLU expert Dr. Walid Saba.
Walid is an old-school AI expert. He is a polymath, a neuroscientist, psychologist, linguist, philosopher, statistician, and logician. He thinks the missing information problem and lack of a typed ontology is the key issue with NLU, not sample efficiency or generalisation. He is a big critic of the deep learning movement and BERTology. We also cover GPT-3 in some detail in today's session, covering Luciano Floridi's recent article "GPT‑3: Its Nature, Scope, Limits, and Consequences" and a commentary on the incredible power of GPT-3 to perform tasks with just a few examples including the Yann LeCun commentary on Facebook and Hackernews.
Time stamps on the YouTube version
0:00:00 Walid intro
00:05:03 Knowledge acquisition bottleneck
00:06:11 Language is ambiguous
00:07:41 Language is not learned
00:08:32 Language is a formal language
00:08:55 Learning from data doesn’t work
00:14:01 Intelligence
00:15:07 Lack of domain knowledge these days
00:16:37 Yannic Kilcher thuglife comment
00:17:57 Deep learning assault
00:20:07 The way we evaluate language models is flawed
00:20:47 Humans do type checking
00:23:02 Ontologic
00:25:48 Comments On GPT3
00:30:54 Yann lecun and reddit
00:33:57 Minds and machines - Luciano
00:35:55 Main show introduction
00:39:02 Walid introduces himself
00:40:20 science advances one funeral at a time
00:44:58 Deep learning obsession syndrome and inception
00:46:14 BERTology / empirical methods are not NLU
00:49:55 Pattern recognition vs domain reasoning, is the knowledge in the data
00:56:04 Natural language understanding is about decoding and not compression, it's not learnable.
01:01:46 Intelligence is about not needing infinite amounts of time
01:04:23 We need an explicit ontological structure to understand anything
01:06:40 Ontological concepts
01:09:38 Word embeddings
01:12:20 There is power in structure
01:15:16 Language models are not trained on pronoun disambiguation and resolving scopes
01:17:33 The information is not in the data
01:19:03 Can we generate these rules on the fly? Rules or data?
01:20:39 The missing data problem is key
01:21:19 Problem with empirical methods and lecunn reference
01:22:45 Comparison with meatspace (brains)
01:28:16 The knowledge graph game, is knowledge constructed or discovered
01:29:41 How small can this ontology of the world be?
01:33:08 Walids taxonomy of understanding
01:38:49 The trend seems to be, less rules is better not the othe way around?
01:40:30 Testing the latest NLP models with entailment
01:42:25 Problems with the way we evaluate NLP
01:44:10 Winograd Schema challenge
01:45:56 All you need to know now is how to build neural networks, lack of rigour in ML research
01:50:47 Is everything learnable
01:53:02 How should we elevate language systems?
01:54:04 10 big problems in language (missing information)
01:55:59 Multiple inheritance is wrong
01:58:19 Language is ambiguous
02:01:14 How big would our world ontology need to be?
02:05:49 How to learn more about NLU
02:09:10 AlphaGo
Walid's blog: https://medium.com/@ontologik
LinkedIn: https://www.linkedin.com/in/walidsaba/