AI:AM #3: Zvi on Fable, the Cases For & Against the Ban, + AI for Math, Logistics & More
Zvi Mowshowitz joins AI in the AM to unpack Anthropic's Fable system card, including its FrontierMath leap, troubling Vending-Bench behavior, decision-theory drift, and signs that model reasoning may be becoming harder to read. The episode then turns to the US government's attempted export-control action against Fable, with Zvi arguing that the cited jailbreak demonstration did not prove the claimed threat while still faulting Anthropic's political handling. Sam Hammond and Judd Rosenblatt add competing reads on state capacity, CAISI, NSA-driven caution, and the alignment world's failure to build trust across partisan lines. The stakes are whether frontier AI capability, safety evaluation, and government power can be coordinated before medicine, mathematics, software, and cyber-relevant systems move further ahead.
For full show notes, links, and references, read the episode page:
https://www.cognitiverevolution.ai/ai-am-3-zvi-on-fable-the-cases-for-against-the-ban-ai-for-math-logistics-more/
Mercury: Command is Mercury’s new conversational interface, giving you natural-language access to your finances and helping you take actions within your existing permissions and approval policies. Visit https://mercury.com to learn more and apply online in minutes.
Sponsor:
Claude:
Claude by Anthropic is an AI collaborator that understands your workflow and helps you tackle research, writing, coding, and organization with deep context. Get started with Claude and explore Claude Pro at https://claude.ai/tcr
CHAPTERS:
(00:00) About the Episode
(01:28) Special Sponsor
(03:17) Weekly highlights preview
(05:23) Fable capability alarms
(16:29) Anthropic government strategy (Part 1)
(16:34) Sponsor: Claude
(18:26) Anthropic government strategy (Part 2)
(27:16) Cyber ban rationale
(37:14) Government power politics
(48:57) Unavoidable control risks
(01:01:42) Government mechanics and empathy
(01:12:50) Legal authority limits
(01:19:02) Pause Overton window
(01:31:58) Medicine, math, safety
(01:47:27) Software without code
(02:01:19) Enterprise world models
(02:10:46) Episode Outro
(02:13:39) Outro
PRODUCED BY:
https://aipodcast.ing
SOCIAL LINKS:
Website: https://www.cognitiverevolution.ai
Twitter (Podcast): https://x.com/cogrev_podcast
Twitter (Nathan): https://x.com/labenz
LinkedIn: https://linkedin.com/in/nathanlabenz/
Youtube: https://youtube.com/@CognitiveRevolutionPodcast
Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431
Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk
Biologically Inspired AI Alignment & Neglected Approaches to AI Safety, with Judd Rosenblatt and Mike Vaiana of AE Studio
In this episode of The Cognitive Revolution, Nathan explores unconventional approaches to AI safety with Judd Rosenblatt and Mike Vaiana from AE Studio. Discover how this innovative company pivoted from brain-computer interfaces to groundbreaking AI alignment research, producing two notable results in cooperative and less deceptive AI systems. Join us for a deep dive into biologically-inspired approaches that offer hope for solving critical AI safety challenges.
Self-Modeling: https://arxiv.org/abs/2407.10188
Self-Other Distinction Minimization: https://www.alignmentforum.org/posts/hzt9gHpNwA2oHtwKX/self-other-overlap-a-neglected-approach-to-ai-alignment
Neglected approaches blog post: https://www.lesswrong.com/posts/qAdDzcBuDBLexb4fC/the-neglected-approaches-approach-ae-studio-s-alignment
Apply to join over 400 Founders and Execs in the Turpentine Network: https://www.turpentinenetwork.co/
SPONSORS:
WorkOS: Building an enterprise-ready SaaS app? WorkOS has got you covered with easy-to-integrate APIs for SAML, SCIM, and more. Join top startups like Vercel, Perplexity, Jasper & Webflow in powering your app with WorkOS. Enjoy a free tier for up to 1M users! Start now at https://bit.ly/WorkOS-Turpentine-Network
Weights & Biases Weave: Weights & Biases Weave is a lightweight AI developer toolkit designed to simplify your LLM app development. With Weave, you can trace and debug input, metadata and output with just 2 lines of code. Make real progress on your LLM development and visit the following link to get started with Weave today: https://wandb.me/cr
80,000 Hours: 80,000 Hours offers free one-on-one career advising for Cognitive Revolution listeners aiming to tackle global challenges, especially in AI. They connect high-potential individuals with experts, opportunities, and personalized career plans to maximize positive impact. Apply for a free call at https://80000hours.org/cognitiverevolution to accelerate your career and contribute to solving pressing AI-related issues.
Omneky: Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off https://www.omneky.com/
RECOMMENDED PODCAST:
This Won't Last - Eavesdrop on Keith Rabois, Kevin Ryan, Logan Bartlett, and Zach Weinberg's monthly backchannel ft their hottest takes on the future of tech, business, and venture capital.
Spotify: https://open.spotify.com/show/2HwSNeVLL1MXy0RjFPyOSz
CHAPTERS:
(00:00:00) About the Show
(00:00:22) Sponsors: WorkOS
(00:01:22) About the Episode
(00:05:18) Introduction and AE Studio Background
(00:11:37) Keys to Success in Building AE Studio
(00:16:57) Sponsors: Weights & Biases Weave | 80,000 Hours
(00:19:37) Universal Launcher and Productivity Gains
(00:24:44) 100x Productivity Increase Explanation
(00:31:46) Brain-Computer Interface and AI Alignment
(00:38:05) Sponsors: Omneky
(00:38:30) Current State of NeuroTech
(00:44:00) Survey on Neglected Approaches in AI Alignment
(00:50:41) Self-Modeling and Biological Inspiration
(00:57:48) Technical Details of Self-Modeling
(01:06:17) Self-Other Distinction Minimization
(01:12:44) Implementation in Language Models
(01:19:00) Compute Costs and Scaling Considerations
(01:24:27) Consciousness Concerns and Future Work
(01:40:24) Evaluating Neglected Approaches
(01:55:56) Closing Thoughts and Policy Considerations
(01:59:25) Outro
