AI Trends 2026: OpenClaw Agents, Reasoning LLMs, and More with Sebastian Raschka - #762
In this episode, Sebastian Raschka, independent LLM researcher and author, joins us to break down how the LLM landscape has changed over the past year and what is likely to matter most in 2026. We discuss the shift from raw model scaling to reasoning-focused post-training, inference-time techniques, and better tool integration. Sebastian explains why methods like self-consistency, self-refinement, and verifiable-reward reinforcement learning have become central to progress in domains like math and coding, and where those approaches still fall short. We also explore agentic workflows in practice, including where multi-agent systems add real value and where reliability constraints still dominate system design. The conversation covers architecture trends such as mixture-of-experts, attention efficiency strategies, and the practical impact of long-context models, alongside persistent challenges like continual learning. We close with Sebastian’s perspective on maintaining strong coding fundamentals in the age of AI assistants and a preview of his new book, Build A Reasoning Model (From Scratch).
The complete show notes for this episode can be found at https://twimlai.com/go/762.
[LIVE] Anthropic Distillation & How Models Cheat (SWE-Bench Dead) | Nathan Lambert & Sebastian Raschka
Swyx joined SAIL! Thank you SAIL Media, Prof. Tom Yeh, 8Lee, Hamid Bagheri, c9n, and many others for tuning into SAIL Live #6 with Nathan Lambert and Sebastian Raschka, PhD. Sharing here for the LS paid subscribers.
We covered:
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.latent.space/subscribe
#490 – State of AI in 2026: LLMs, Coding, Scaling Laws, China, Agents, GPUs, AGI
Nathan Lambert and Sebastian Raschka are machine learning researchers, engineers, and educators. Nathan is the post-training lead at the Allen Institute for AI (Ai2) and the author of The RLHF Book. Sebastian Raschka is the author of Build a Large Language Model (From Scratch) and Build a Reasoning Model (From Scratch).
Thank you for listening ❤ Check out our sponsors: https://lexfridman.com/sponsors/ep490-sc
See below for timestamps, transcript, and to give feedback, submit questions, contact Lex, etc.
Transcript:
https://lexfridman.com/ai-sota-2026-transcript
CONTACT LEX:
Feedback – give feedback to Lex: https://lexfridman.com/survey
AMA – submit questions, videos or call-in: https://lexfridman.com/ama
Hiring – join our team: https://lexfridman.com/hiring
Other – other ways to get in touch: https://lexfridman.com/contact
SPONSORS:
To support this podcast, check out our sponsors & get discounts:
Box: Intelligent content management platform.
Go to https://box.com/ai
Quo: Phone system (calls, texts, contacts) for businesses.
Go to https://quo.com/lex
UPLIFT Desk: Standing desks and office ergonomics.
Go to https://upliftdesk.com/lex
Fin: AI agent for customer service.
Go to https://fin.ai/lex
Shopify: Sell stuff online.
Go to https://shopify.com/lex
CodeRabbit: AI-powered code reviews.
Go to https://coderabbit.ai/lex
LMNT: Zero-sugar electrolyte drink mix.
Go to https://drinkLMNT.com/lex
Perplexity: AI-powered answer engine.
Go to https://perplexity.ai/
OUTLINE:
(00:00) – Introduction
(01:39) – Sponsors, Comments, and Reflections
(16:29) – China vs US: Who wins the AI race?
(25:11) – ChatGPT vs Claude vs Gemini vs Grok: Who is winning?
(36:11) – Best AI for coding
(43:02) – Open Source vs Closed Source LLMs
(54:41) – Transformers: Evolution of LLMs since 2019
(1:02:38) – AI Scaling Laws: Are they dead or still holding?
(1:18:45) – How AI is trained: Pre-training, Mid-training, and Post-training
(1:51:51) – Post-training explained: Exciting new research directions in LLMs
(2:12:43) – Advice for beginners on how to get into AI development & research
(2:35:36) – Work culture in AI (72+ hour weeks)
(2:39:22) – Silicon Valley bubble
(2:43:19) – Text diffusion models and other new research directions
(2:49:01) – Tool use
(2:53:17) – Continual learning
(2:58:39) – Long context
(3:04:54) – Robotics
(3:14:04) – Timeline to AGI
(3:21:20) – Will AI replace programmers?
(3:39:51) – Is the dream of AGI dying?
(3:46:40) – How AI will make money?
(3:51:02) – Big acquisitions in 2026
(3:55:34) – Future of OpenAI, Anthropic, Google DeepMind, xAI, Meta
(4:08:08) – Manhattan Project for AI
(4:14:42) – Future of NVIDIA, GPUs, and AI compute clusters
(4:22:48) – Future of human civilization
767: Open-Source LLM Libraries and Techniques, with Dr. Sebastian Raschka
Jon Krohn sits down with Sebastian Raschka to discuss his latest book, Machine Learning Q and AI, the open-source libraries developed by Lightning AI, how to exploit the greatest opportunities for LLM development, and what’s on the horizon for LLMs.
This episode is brought to you by the DataConnect Conference, and by Data Universe, the out-of-this-world data conference. Interested in sponsoring a SuperDataScience Podcast episode? Visit passionfroot.me/superdatascience for sponsorship information.
In this episode you will learn:
• All about Machine Learning Q and AI [04:13]
• Sebastian Raschka’s role as Staff Research Engineer at Lightning AI [19:21]
• PyTorch Lightning’s and Lightning Fabric’s capabilities [39:32]
• Large language models: Opportunities and challenges [43:35]
• DoRA vs LoRA [48:56]
• How to be a successful AI educator [1:34:18]
Additional materials: www.superdatascience.com/767
A former Foxconn executive tries to explain what went wrong in Wisconsin
Alan Yeung is a professor of entrepreneurship at the University of Wisconsin-Madison and the former head of the Foxconn project in Wisconsin. If you don’t quite remember, the Foxconn project in Wisconsin was announced in 2017 as a massive deal to build the first “Generation 10.5” LCD factory in North America. It was also one of the first big moments in the Trump presidency, complete with President Trump holding a golden shovel at a lavish groundbreaking ceremony where he said the factory would be “the eighth wonder of the world.”
But it turned out that while Foxconn was putting on a great show, no LCD factory was actually getting built, even though Foxconn kept saying it was happening.
Links
We're nominated for a Webby! Vote for Decoder!
The award winning story from Josh Dzieza - The 8th wonder of the world
Wisconsin's $4.1 billion Foxconn factory boondoggle
Foxconn’s $100M deal with the University of Wisconsin has students worried
What a new governor means for Wisconsin’s controversial Foxconn factory
Foxconn and the village: the $10B factory deal that turned one small Wisconsin town upside down
No one seems to know what Foxconn is doing in Wisconsin
After a ‘personal conversation’ with Trump, Foxconn says it will build a factory in Wisconsin after all
Foxconn is confusing the hell out of Wisconsin
Foxconn promised a ‘correction’ about empty buildings in Wisconsin two weeks ago, and it hasn’t said a word since
With Foxconn chief’s Trump meeting, the Wisconsin project gets even more political
One month ago, Foxconn said its innovation centers weren’t empty — they still are
Foxconn’s delays might finally give Wisconsin the upper hand
One year after Trump’s Foxconn groundbreaking, there is almost nothing to show for it
Even fixing Wisconsin’s Foxconn deal won’t fix it, says state-requested report
Foxconn’s first announced product for its Wisconsin factory is an airport coffee robot
Foxconn releases and immediately cancels plans for a giant dome in Wisconsin
Foxconn's giant glass dome in Wisconsin is back, baby
Exclusive: documents show Foxconn refuses to renegotiate Wisconsin deal
Foxconn’s buildings in Wisconsin are still empty, one year later
Exclusive: Wisconsin denies Foxconn tax subsidies after contract negotiations fail
The 8th wonder of the world
Exclusive: Wisconsin report confirms Foxconn's “LCD factory” isn't real
Foxconn tells Wisconsin it never promised to build an LCD factory
Intel selects Ohio for ‘largest silicon manufacturing location on the planet’
Transcript:
https://www.theverge.com/e/22794506
Credits:
Decoder is a production of The Verge, and part of the Vox Media Podcast Network.
Today’s episode was produced by Creighton DeSimone and Jackie McDermott and it was edited by Callie Wright.
The Decoder music is by Breakmaster Cylinder. Our Sr Audio Director is Andrew Marino and our Executive Producer is Eleanor Donovan.
Learn more about your ad choices. Visit podcastchoices.com/adchoices
Advancing Hands-On Machine Learning Education with Sebastian Raschka - #565
Today we’re joined by Sebastian Raschka, an assistant professor at the University of Wisconsin-Madison and lead AI educator at Grid.ai. In our conversation with Sebastian, we explore his work around AI education, including the “hands-on” philosophy that he takes when building these courses, his recent book Machine Learning with PyTorch and Scikit-Learn, his advise to beginners in the field when they’re trying to choose tools and frameworks, and more.
We also discuss his work on Pytorch Lightning, a platform that allows users to organize their code and integrate it into other technologies, before switching gears and discuss his recent research efforts around ordinal regression, including a ton of great references that we’ll link on the show notes page below!
The complete show notes for this episode can be found at twimlai.com/go/565
#190 – Jordan Ellenberg: Mathematics of High-Dimensional Shapes and Geometries
Jordan Ellenberg is a mathematician and author of Shape and How Not to Be Wrong. Please support this podcast by checking out our sponsors:
– Secret Sauce: https://wondery.com/shows/secret-sauce/
– ExpressVPN: https://expressvpn.com/lexpod and use code LexPod to get 3 months free
– Blinkist: https://blinkist.com/lex and use code LEX to get 25% off premium
– Indeed: https://indeed.com/lex to get $75 credit
EPISODE LINKS:
Jordan’s Website: http://www.jordanellenberg.com
Jordan’s Twitter: https://twitter.com/JSEllenberg
PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
YouTube Full Episodes: https://youtube.com/lexfridman
YouTube Clips: https://youtube.com/lexclips
SUPPORT & CONNECT:
– Check out the sponsors above, it’s the best way to support this podcast
– Support on Patreon: https://www.patreon.com/lexfridman
– Twitter: https://twitter.com/lexfridman
– Instagram: https://www.instagram.com/lexfridman
– LinkedIn: https://www.linkedin.com/in/lexfridman
– Facebook: https://www.facebook.com/lexfridman
– Medium: https://medium.com/@lexfridman
OUTLINE:
Here’s the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time.
(00:00) – Introduction
(06:44) – Mathematical thinking
(10:21) – Geometry
(14:58) – Symmetry
(25:29) – Math and science in the Soviet Union
(33:09) – Topology
(47:57) – Do we live in many more than 4 dimensions?
(52:28) – How many holes does a straw have
(1:01:53) – 3Blue1Brown
(1:07:40) – Will AI ever win a Fields Medal?
(1:16:05) – Fermat’s last theorem
(1:33:23) – Reality cannot be explained simply
(1:39:08) – Prime numbers
(2:00:37) – John Conway’s Game of Life
(2:12:29) – Group theory
(2:15:45) – Gauge theory
(2:23:47) – Grigori Perelman and the Poincare Conjecture
(2:33:59) – How to learn math
(2:41:08) – Advice for young people
(2:43:13) – Meaning of life
High-Dimensional Robust Statistics with Ilias Diakonikolas - #351
Today we’re joined by Ilias Diakonikolas, faculty in the CS department at the University of Wisconsin-Madison, and author of the paper Distribution-Independent PAC Learning of Halfspaces with Massart Noise, recipient of the NeurIPS 2019 Outstanding Paper award. The paper is regarded as the first progress made around distribution-independent learning with noise since the 80s. In our conversation, we explore robustness in ML, problems with corrupt data in high-dimensional settings, and of course, the paper.