University of Oxford

Economics of AGI episode w Alex Imas and Phil Trammell. There’s a bunch of important questions about how we deal with AI that only economics can answer. What is the optimal way to tax and redistribute the wealth that will be generated? How should countries not in the AI supply chain index into the gains? Is there any world where inequality doesn’t explode? It might seem like these questions have obvious answers, but the first thing economics teaches you is that your intuitions can often be entirely wrong. It was very helpful to chat through these things with Alex and Phil. Watch on YouTube; read the transcript. Sponsors Jane Street invests heavily in turning smart people into exceptional researchers and engineers. In addition to their apprenticeship model, Jane Street runs lectures and bootcamps in their in-office classrooms -- managers clear their teams’ schedules to encourage attendance. If you’d like to work at a place that takes learning this seriously, Jane Street is hiring. Check out their open roles at janestreet.com/dwarkesh Google’s Gemini Omni has incredible video editing capabilities -- you can upload a video and have Omni change the background, adjust lighting, or add specific elements. But Omni is also a preview of how future frontier models will be trained -- fully multimodal on both input and output. You can try it yourself in the Gemini app at gemini.google or in Flow at flow.google Cursor used targeted RL with textual feedback to help train their Composer 2.5 model. One of their researchers, Sasha Rush, gave me an impromptu blackboard lecture to explain how this form of on-policy self-distillation works -- I posted the full thing on X. If you want to try Composer 2.5, go to cursor.com/dwarkesh Timestamps (00:00:00) – Will capital share increase? (00:19:36) – Messy Middle scenario (00:25:57) – How to tax and redistribute AI wealth (00:30:02) – Why demand collapse is unlikely (00:39:26) – Human employees would be hard to integrate into the machine economy (00:43:08) – What if some humans (or AIs) value wealth accumulation intrinsically? (01:01:28) – What should developing countries do? Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe

Alex Imas

Professor of Economics

University of ChicagoUniversity of Chicago Booth School of Business

Phil Trammell

Researcher

University of Oxford

Carissa Véliz is an Oxford philosopher and the author of Prophecy: Prediction, Power, and the Fight for the Future, from Ancient Oracles to AI. Véliz joins Big Technology Podcast to discuss whether society has become dangerously naive about prediction as AI systems shape decisions around jobs, loans, justice, surveillance, and war. Tune in to hear a debate about predictive algorithms, generative AI, prediction markets, and whether forecasts are actually tools of knowledge or instruments of power. We also cover privacy, policing, protest anonymity, flood prediction, and why humor might be one of the best defenses against a prediction-obsessed world. --- Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice. More on Carissa: https://www.chartwellspeakers.com/speaker/carissa-veliz-2/ Find the book: https://www.amazon.com/Prophecy-Prediction-Future-Ancient-Oracles/dp/0385550979 Want a discount for Big Technology on Substack + Discord? Here’s 25% off for the first year: https://www.bigtechnology.com/subscribe?coupon=0843016b Learn more about your ad choices. Visit megaphone.fm/adchoices

Carissa Véliz

Professor / Author

University of Oxford

What happens when your team doesn’t trust each other? How can you repair damaged trust at work? How has technology changed the way we decide who to trust? Author and trust researcher Rachel Botsman joins host Jeff Berman to tackle these questions and more. Her latest book is How to Trust and Be Trusted: https://www.rachelbotsman.com/work/how-to-trust-and-be-trusted See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

Rachel Botsman

Author and Trust Researcher

University of Oxford

Tech innovation’s dependence on economic systems, trust in technology throughout history, and job displacement through AI: The Dieter Schwartz Associate Professor of AI and work at the University of Oxford, Carl Benedikt Frey, talks to Jon Krohn about his latest book, How Progress Ends, as well as how different economic systems deal with innovation and scaling, dealing with the homogeneity of generative AI output, and how to stay afloat in the new wave of job automation. This episode is brought to you by the ⁠Dell⁠, by ⁠Intel⁠, by ⁠ODSC, the Open Data Science Conference⁠ and by ⁠Gurobi⁠. Additional materials: ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠www.superdatascience.com/925⁠⁠⁠⁠⁠ Interested in sponsoring a SuperDataScience Podcast episode? Email natalie@superdatascience.com for sponsorship information. In this episode you will learn: (04:00) All about How Progress Ends: Technology, Innovation, and the Fate of Nations (14:26) The role of weak ties in driving technological innovation (18:22) How to keep innovating as a big business (48:05) What we can learn and apply from previous industrial revolutions (54:33) How workers can try to ‘future-proof’ themselves

Carl Benedikt Frey

Dieter Schwartz Associate Professor of AI and Work

University of OxfordOxford Internet Institute

We interview Professor Christopher Summerfield from Oxford University about his new book "These Strange New Minds: How AI Learned to Talk and What It". AI learned to understand the world just by reading text - something scientists thought was impossible. You don't need to see a cat to know what one is; you can learn everything from words alone. This is "the most astonishing scientific discovery of the 21st century."People are split: some refuse to call what AI does "thinking" even when it outperforms humans, while others believe if it acts intelligent, it is intelligent. Summerfield takes the middle ground - AI does something genuinely like human reasoning, but that doesn't make it human.Sponsor messages:========Google Gemini: Google Gemini features Veo3, a state-of-the-art AI video generation model in the Gemini app. Sign up at https://gemini.google.comTufa AI Labs are hiring for ML Engineers and a Chief Scientist in Zurich/SF. They are top of the ARCv2 leaderboard! https://tufalabs.ai/========Prof. Christopher Summerfieldhttps://www.psy.ox.ac.uk/people/christopher-summerfieldThese Strange New Minds: How AI Learned to Talk and What It Meanshttps://amzn.to/4e26BVaTable of Contents:Introduction & Setup00:00:00 Superman 3 Metaphor - Humans Absorbed by Machines00:02:01 Book Introduction & AI Debate Context00:03:45 Sponsor Segments (Google Gemini, Tufa Labs)Philosophical Foundations00:04:48 The Fractured AI Discourse00:08:21 Ancient Roots: Aristotle vs Plato (Empiricism vs Rationalism)00:10:14 Historical AI: Symbolic Logic and Its LimitsThe Language Revolution00:12:11 ChatGPT as the Rubicon Moment00:14:00 The Astonishing Discovery: Learning Reality from Words Alone00:15:47 Equivalentists vs Exceptionalists DebateCognitive Science Perspectives00:19:12 Functionalism and the Duck Test00:21:48 Brain-AI Similarities and Computational Principles00:24:53 Reconciling Chomsky: Evolution vs Learning00:28:15 Lamarckian AI vs Darwinian Human LearningThe Reality of AI Capabilities00:30:29 Anthropomorphism and the Clever Hans Effect00:32:56 The Intentional Stance and Nature of Thinking00:37:56 Three Major AI Worries: Agency, Personalization, DynamicsSocietal Risks and Complex Systems00:37:56 AI Agents and Flash Crash Scenarios00:42:50 Removing Frictions: The Lawfare Example00:46:15 Gradual Disempowerment Theory00:49:18 The Faustian Pact of TechnologyHuman Agency and Control00:51:18 The Crisis of Authenticity00:56:22 Psychology of Control vs Reward01:00:21 Dopamine Hacking and Variable ReinforcementFuture Directions01:02:27 Evolution as Goal-less Optimization01:03:31 Open-Endedness and Creative Evolution01:06:46 Writing, Creativity, and AI-Generated Content01:08:18 Closing RemarksREFS:Academic References (Abbreviated)Essential Books"These Strange New Minds" - C. Summerfield [00:02:01] - Main discussion topic"The Mind is Flat" - N. Chater [00:33:45] - Summerfield's favorite on cognitive illusions"AI: A Guide for Thinking Humans" - M. Mitchell [00:04:58] - Host's previous favorite"Principia Mathematica" - Russell & Whitehead [00:11:00] - Logic Theorist reference"Syntactic Structures" - N. Chomsky (1957) [00:13:30] - Generative grammar foundation"Why Greatness Cannot Be Planned" - Stanley & Lehman [01:04:00] - Open-ended evolutionKey Papers & Studies"Gradual Disempowerment" - D. Duvenaud [00:46:45] - AI threat model"Counterfeit People" - D. Dennett (Atlantic) [00:52:45] - AI societal risks"Open-Endedness is Essential..." - DeepMind/Rocktäschel/Hughes [01:03:42]Heider & Simmel (1944) [00:30:45] - Agency attribution to shapesWhitehall Studies - M. Marmot [00:59:32] - Control and health outcomes"Clever Hans" - O. Pfungst (1911) [00:31:47] - Animal intelligence illusionHistorical References <trunc, see https://youtu.be/35r0iSajXjA>

Christopher Summerfield

Professor

University of Oxford

Sam Harris speaks with David Deutsch about quantum physics and current events. They discuss the "many-worlds" interpretation of QM, Schrödinger's cat, constructor theory, quantum computing and whether it will ever be practically possible, recent developments in AI, the prospects of artificial super-intelligence, the alignment problem, antisemitism and the historical persecution of Jews, misconceptions about Israel, the future of the Jews in Israel and the West, and other topics. If the Making Sense podcast logo in your player is BLACK, you can SUBSCRIBE to gain access to all full-length episodes at samharris.org/subscribe. Learning how to train your mind is the single greatest investment you can make in life. That's why Sam Harris created the Waking Up app. From rational mindfulness practice to lessons on some of life's most important topics, join Sam as he demystifies the practice of meditation and explores the theory behind it.

David Deutsch

Physicist & Author

University of Oxford

Prof. Jakob Foerster, a leading AI researcher at Oxford University and Meta, and Chris Lu, a researcher at OpenAI -- they explain how AI is moving beyond just mimicking human behaviour to creating truly intelligent agents that can learn and solve problems on their own. Foerster champions open-source AI for responsible, decentralised development. He addresses AI scaling, goal misalignment (Goodhart's Law), and the need for holistic alignment, offering a quick look at the future of AI and how to guide it. SPONSOR MESSAGES: *** CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments. Check out their super fast DeepSeek R1 hosting! https://centml.ai/pricing/ Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on o-series style reasoning and AGI. They are hiring a Chief Engineer and ML engineers. Events in Zurich. Goto https://tufalabs.ai/ *** TRANSCRIPT/REFS: https://www.dropbox.com/scl/fi/yqjszhntfr00bhjh6t565/JAKOB.pdf?rlkey=scvny4bnwj8th42fjv8zsfu2y&dl=0 Prof. Jakob Foerster https://x.com/j_foerst https://www.jakobfoerster.com/ University of Oxford Profile: https://eng.ox.ac.uk/people/jakob-foerster/ Chris Lu: https://chrislu.page/ TOC 1. GPU Acceleration and Training Infrastructure [00:00:00] 1.1 ARC Challenge Criticism and FLAIR Lab Overview [00:01:25] 1.2 GPU Acceleration and Hardware Lottery in RL [00:05:50] 1.3 Data Wall Challenges and Simulation-Based Solutions [00:08:40] 1.4 JAX Implementation and Technical Acceleration 2. Learning Frameworks and Policy Optimization [00:14:18] 2.1 Evolution of RL Algorithms and Mirror Learning Framework [00:15:25] 2.2 Meta-Learning and Policy Optimization Algorithms [00:21:47] 2.3 Language Models and Benchmark Challenges [00:28:15] 2.4 Creativity and Meta-Learning in AI Systems 3. Multi-Agent Systems and Decentralization [00:31:24] 3.1 Multi-Agent Systems and Emergent Intelligence [00:38:35] 3.2 Swarm Intelligence vs Monolithic AGI Systems [00:42:44] 3.3 Democratic Control and Decentralization of AI Development [00:46:14] 3.4 Open Source AI and Alignment Challenges [00:49:31] 3.5 Collaborative Models for AI Development REFS [[00:00:05] ARC Benchmark, Chollet https://github.com/fchollet/ARC-AGI [00:03:05] DRL Doesn't Work, Irpan https://www.alexirpan.com/2018/02/14/rl-hard.html [00:05:55] AI Training Data, Data Provenance Initiative https://www.nytimes.com/2024/07/19/technology/ai-data-restrictions.html [00:06:10] JaxMARL, Foerster et al. https://arxiv.org/html/2311.10090v5 [00:08:50] M-FOS, Lu et al. https://arxiv.org/abs/2205.01447 [00:09:45] JAX Library, Google Research https://github.com/jax-ml/jax [00:12:10] Kinetix, Mike and Michael https://arxiv.org/abs/2410.23208 [00:12:45] Genie 2, DeepMind https://deepmind.google/discover/blog/genie-2-a-large-scale-foundation-world-model/ [00:14:42] Mirror Learning, Grudzien, Kuba et al. https://arxiv.org/abs/2208.01682 [00:16:30] Discovered Policy Optimisation, Lu et al. https://arxiv.org/abs/2210.05639 [00:24:10] Goodhart's Law, Goodhart https://en.wikipedia.org/wiki/Goodhart%27s_law [00:25:15] LLM ARChitect, Franzen et al. https://github.com/da-fr/arc-prize-2024/blob/main/the_architects.pdf [00:28:55] AlphaGo, Silver et al. https://arxiv.org/pdf/1712.01815.pdf [00:30:10] Meta-learning, Lu, Towers, Foerster https://direct.mit.edu/isal/proceedings-pdf/isal2023/35/67/2354943/isal_a_00674.pdf [00:31:30] Emergence of Pragmatics, Yuan et al. https://arxiv.org/abs/2001.07752 [00:34:30] AI Safety, Amodei et al. https://arxiv.org/abs/1606.06565 [00:35:45] Intentional Stance, Dennett https://plato.stanford.edu/entries/ethics-ai/ [00:39:25] Multi-Agent RL, Zhou et al. https://arxiv.org/pdf/2305.10091 [00:41:00] Open Source Generative AI, Foerster et al. https://arxiv.org/abs/2405.08597 <trunc, see PDF/YT>

Jakob Foerster

Professor

University of Oxford

Chris Lu

Researcher

Sakana AIOpenAI

On Sept. 4, 2024, Me, Myself, and AI host Sam Ransbotham moderated a panel discussion at a Georgetown University/World Bank event, Jobs in the Age of AI. Afterward, he interviewed keynote speaker Carl Benedikt Frey, Dieter Schwarz Associate Professor of AI and Work at the Oxford Internet Institute, and panelist Karin Kimbrough, LinkedIn’s chief economist. In this bonus episode recorded during this discussion, hear from Frey and Kimbrough about how artificial intelligence is impacting workers, labor trends, and the economy. Read the episode transcript here. For further information: Watch sessions from the AI in Action event on demand. Access on-demand recordings from all prior AI in Action events. Read event organizers Timothy DeStefano and Jonathan Timmis’s paper, “Do Capital Incentives Distort Technology Diffusion? Evidence on Cloud, Big Data, and AI.” Me, Myself, and AI is a collaborative podcast from MIT Sloan Management Review and Boston Consulting Group and is hosted by Sam Ransbotham and Shervin Khodabandeh. Our engineer is David Lishansky, and the coordinating producers are Allison Ryder and Alanna Hooper. Stay in touch with us by joining our LinkedIn group, AI for Leaders at mitsmr.com/AIforLeaders or by following Me, Myself, and AI on LinkedIn. We encourage you to rate and review our show. Your comments may be used in Me, Myself, and AI materials.

Carl Benedikt Frey

Dieter Schwartz Associate Professor of AI and Work

University of OxfordOxford Internet Institute

Karin Kimbrough

Chief Economist

LinkedIn

Sam Harris speaks with Nick Bostrom about ongoing progress in artificial intelligence. They discuss the twin concerns about the failure of alignment and the failure to make progress, why smart people don't perceive the risk of superintelligent AI, the governance risk, path dependence and "knotty problems," the idea of a solved world, Keynes's predictions about human productivity, the uncanny valley of utopia, the replacement of human labor and other activities, meaning and purpose, digital isolation and plugging into something like the Matrix, pure hedonism, the asymmetry between pleasure and pain, increasingly subtle distinctions in experience, artificial purpose, altering human values at the level of the brain, ethical changes in the absence of extreme suffering, our cosmic endowment, longtermism, problems with consequentialism, the ethical conundrum of dealing with small probabilities of large outcomes, and other topics. If the Making Sense podcast logo in your player is BLACK, you can SUBSCRIBE to gain access to all full-length episodes at samharris.org/subscribe. Learning how to train your mind is the single greatest investment you can make in life. That's why Sam Harris created the Waking Up app. From rational mindfulness practice to lessons on some of life's most important topics, join Sam as he demystifies the practice of meditation and explores the theory behind it.

Nick Bostrom

Philosopher and Director

Future of Humanity InstituteUniversity of Oxford

In this special cross-post episode of The Cognitive Revolution, Nathan shares a fascinating conversation between Spencer Greenberg and philosopher Nick Bostrom from the Clearer Thinking podcast. They explore Bostrom's latest book, "Deep Utopia," and discuss the challenges of envisioning a truly desirable future. Discover how advanced AI could reshape our concept of purpose and meaning, and hear thought-provoking ideas on finding fulfillment in a world where technology solves our pressing problems. Join us for an insightful journey into the potential evolution of human flourishing and the quest for positive visions of the future. Originally appeared in Clearer Thinking Podcast: https://podcast.clearerthinking.org/episode/224/nick-bostrom-the-path-to-utopia Check out the Clearer Thinking with Spencer Greenberg Podcast here: https://podcast.clearerthinking.org/ Deep Utopia Book: https://www.amazon.com/Deep-Utopia-Meaning-Solved-World/dp/1646871642/ Apply to join over 400 Founders and Execs in the Turpentine Network: https://www.turpentinenetwork.co/ SPONSORS: Oracle: Oracle Cloud Infrastructure (OCI) is a single platform for your infrastructure, database, application development, and AI needs. OCI has four to eight times the bandwidth of other clouds; offers one consistent price, and nobody does data better than Oracle. If you want to do more and spend less, take a free test drive of OCI at https://oracle.com/cognitive Brave: The Brave search API can be used to assemble a data set to train your AI models and help with retrieval augmentation at the time of inference. All while remaining affordable with developer first pricing, integrating the Brave search API into your workflow translates to more ethical data sourcing and more human representative data sets. Try the Brave search API for free for up to 2000 queries per month at https://bit.ly/BraveTCR Omneky: Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off https://www.omneky.com/ Weights & Biases Weave: Weights & Biases Weave is a lightweight AI developer toolkit designed to simplify your LLM app development. With Weave, you can trace and debug input, metadata and output with just 2 lines of code. Make real progress on your LLM development and visit the following link to get started with Weave today: https://wandb.me/cr RECOMMENDED PODCAST: This Won't Last. Eavesdrop on Keith Rabois, Kevin Ryan, Logan Bartlett, and Zach Weinberg's monthly backchannel. They unpack their hottest takes on the future of tech, business, venture, investing, and politics. Apple Podcasts: https://podcasts.apple.com/us/podcast/id1765665937 Spotify: https://open.spotify.com/show/2HwSNeVLL1MXy0RjFPyOSz YouTube: https://www.youtube.com/@ThisWontLastpodcast CHAPTERS: (00:00:00) About the Show (00:00:22) About the Episode (00:02:58) Introduction to the podcast (00:03:26) Dystopias vs utopias in fiction (00:07:29) Material abundance and utopia (00:14:57) AI and the future of work (00:20:10) AI companions and human relationships (00:22:57) Sponsors: Weights & Biases Weave | Oracle (00:25:01) Sponsor message: Positly research platform (00:26:04) Surveillance and global coordination (00:44:38) Sponsors: Omneky | Brave (00:44:52) Sponsor message: Transparent Replications project (00:46:07) AI governance challenges (00:49:36) Deep Utopia book's purpose (00:53:09) Global coordination strategies (00:59:13) The vulnerable world hypothesis (01:05:18) Bostrom's meta-ethical views (01:08:32) Listener question on meditation (01:10:17) Outro

Nick Bostrom

Philosopher and Director

Future of Humanity InstituteUniversity of Oxford

Nick Bostrom is a renowned philosopher and bestselling author of "Superintelligence" and "Deep Utopia." He joins Big Technology to discuss the potential outcomes of advanced artificial intelligence, from existential risks to utopian possibilities. Tune in to hear Bostrom's thoughts on how humanity might navigate the transition to a world of superintelligent AI and what life could look like in a technologically "solved" world. We also cover the evolution of AI safety concerns, the concept of effective accelerationism, and the philosophical implications of living in a post-scarcity society. Hit play for a mind-expanding conversation about the future of humanity and the profound challenges and opportunities that lie ahead. --- Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice. For weekly updates on the show, sign up for the pod newsletter on LinkedIn: https://www.linkedin.com/newsletters/6901970121829801984/ Want a discount for Big Technology on Substack? Here’s 40% off for the first year: https://tinyurl.com/bigtechnology Questions? Feedback? Write to: bigtechnologypodcast@gmail.com Learn more about your ad choices. Visit megaphone.fm/adchoices

Nick Bostrom

Philosopher and Director

Future of Humanity InstituteUniversity of Oxford

Nick Bostrom is a philosopher, professor at the University of Oxford and an author For generations, the future of humanity was envisioned as a sleek, vibrant utopia filled with remarkable technological advancements where machines and humans would thrive together. As we stand on the supposed brink of that future, it appears quite different from our expectations. So what does humanity's future actually hold? Expect to learn what it means to live in a perfectly solved world, whether we are more likely heading toward a utopia or a catastrophe, how humans will find a meaning in a world that no longer needs our contributions, what the future of religion could look like, a breakdown of all the different stages we will move through on route to a final utopia, the current state of AI safety & risk and much more... Sponsors: Get a 20% discount on Nomatic’s amazing luggage at https://nomatic.com/modernwisdom (use code MW20) Get up to 70% off Gymshark's Summer Sale at https://gym.sh/modernwisdom (use code MW10) Get a 20% discount & free shipping on your Lawnmower 5.0 at https://manscaped.com/modernwisdom (use code MODERNWISDOM) Timestamps: (00:00) Is Nick Hopeful About AI? (03:20) How We Can Get AI Right (07:07) The Moral Status of Non-Human Intelligences (17:36) Different Types of Utopia (19:38) The Human Experience in a Solved World (31:32) Using AI to Satisfy Human Desires (43:25) Current Things That Would Stay in Utopia (49:54) The Value of Daily Struggles (55:07) Implications of Extreme Human Longevity (01:00:19) Constraints That We Can’t Get Past (01:07:27) How Important is This Time for Humanity’s Future? (01:13:40) Biggest AI Development Surprises (01:21:24) Current State of AI Safety (01:28:06) Where to Find Nick Extra Stuff: Get my free reading list of 100 books to read before you die: https://chriswillx.com/books Try my productivity energy drink Neutonic: https://neutonic.com/modernwisdom Episodes You Might Enjoy: This Is How To Master Your Life - David Goggins - #577: ⁠⁠⁠lnkfi.re/SN-Goggins⁠⁠⁠ How To Destroy Your Negative Beliefs - Dr Jordan Peterson - #712: ⁠⁠⁠lnkfi.re/SN-Peterson⁠⁠⁠ The Secret Tools To Hack Your Brain - Dr Andrew Huberman - #700: ⁠⁠⁠lnkfi.re/SN-Huberman⁠⁠ - Get In Touch: Instagram: https://www.instagram.com/chriswillx Twitter: https://www.twitter.com/chriswillx YouTube: https://www.youtube.com/modernwisdompodcast Email: https://chriswillx.com/contact - Learn more about your ad choices. Visit megaphone.fm/adchoices

Nick Bostrom

Philosopher and Director

Future of Humanity InstituteUniversity of Oxford

This episode is sponsored by Netsuite by Oracle, the number one cloud financial system, streamlining accounting, financial management, inventory, HR, and more. NetSuite is offering a one-of-a-kind flexible financing program. Head to https://netsuite.com/EYEONAI to know more. Venture into the future of AI with Nick Bostrom, a philosopher at the University of Oxford known for his work on existential risk, the anthropic principle, human enhancement ethics, whole brain emulation, superintelligence risks, and the reversal test. On episode #181 of Eye on AI, Nick Bostrom, explores the existential and societal implications of AI reaching and surpassing human capabilities. As we contemplate a world where all tasks are performed by AI, Nick discusses the potential for a 'technologically solved' society and its impact on human purpose and motivation. Join us as Nick provides insights into his latest book, "Deep Utopia," where he questions how humans will find meaning when artificial intelligence handles every aspect of labor and creativity. He elaborates on the risks, ethical considerations, and philosophical dilemmas we face as AI continues to evolve at an unprecedented pace. This episode is an essential exploration of the shifts AI may bring to our societal structures, labour markets, and individual lives. If you find yourself intrigued by the philosophical journey into AI's potential to redefine humanity, hit the like button and subscribe for more thoughtful discussions on the future landscapes shaped by artificial intelligence. Stay Updated: Craig Smith Twitter: https://twitter.com/craigss Eye on A.I. Twitter: https://twitter.com/EyeOn_AI (00:00) Introduction and the Concept of a 'Solved World' (03:05) Nick Bostrom's Background (06:06) Exploring the Anthropomorphism in Modern AI (08:02) Predictions and the 'Hockey Stick' Graph (10:13) AI Safety and Public Perception (12:58) Deep Utopia and the Search for Meaning (15:46) Life in a Technologically Mature World (18:17) Existential Malaise in Modern Society (20:43) The Potential of Technological Maturity (23:51) Philosophical Implications of a Solved World (28:20) Engineering Happiness and Neurological Adjustments (32:18) Remaining Human Tasks and Cultural Values (35:45) The Future of Humanity (47:03) Closing Remarks and Sponsor Message

Nick Bostrom

Philosopher and Director

Future of Humanity InstituteUniversity of Oxford

My guest today is Robin Dunbar. Robin is a biological anthropologist, evolutionary psychologist, and specialist in primate behavior. He is the man behind Dunbar’s number, a theory about the number of stable relationships we can maintain at once. Robin unravels the thread of research that led him to Dunbar’s number and describes how this plays into every single person’s layers of human connection. It was fascinating to hear how his findings on social circles have implications for optimally structuring businesses and organizations, as well as the idea of homophily, all of which Robin thoughtfully explains. It was a treat to get to explore these topics with Robin Dunbar himself so please enjoy this great conversation.   Listen to Founders Podcast For the full show notes, transcript, and links to mentioned content, check out the episode page here. ----- This episode is brought to you by Tegus, the only investment research platform built for fundamental investors. Whether you’re trying to get up to speed on a new market or keep tabs on a portfolio company, Tegus is the end-to-end investment research platform you need. With Tegus, you can quickly understand a company's business model, drivers, benchmarks, and management quality. To monitor an entire market, download our pre-built financial models — or update your own with the latest data using Tegus’ new Excel Add-In. Tegus gives you all of this and more, all bundled into a single software license. Find out why 95% of the top 20 global private equity firms are Tegus customers. Learn more and get your free trial at tegus.com/patrick. ----- Invest Like the Best is a property of Colossus, LLC. For more episodes of Invest Like the Best, visit joincolossus.com/episodes.  Past guests include Tobi Lutke, Kevin Systrom, Mike Krieger, John Collison, Kat Cole, Marc Andreessen, Matthew Ball, Bill Gurley, Anu Hariharan, Ben Thompson, and many more. Stay up to date on all our podcasts by signing up to Colossus Weekly, our quick dive every Sunday highlighting the top business and investing concepts from our podcasts and the best of what we read that week. Sign up here. Follow us on Twitter: @patrick_oshag | @JoinColossus Editing and post-production work for this episode was provided by The Podcast Consultant (https://thepodcastconsultant.com). Show Notes: (00:00:00) Welcome to Invest Like the Best (00:04:39) The Journey to Discovering Dunbar's Number (00:07:28) Exploring the Layers of Human Connection (00:12:27) The Significance of the 1.5 Layer in Social Circles (00:16:55) Surprising Insights from Social Network Studies (00:20:40) Applying Dunbar's Number to Organizational Structures (00:27:51) The Science of Social Bonding in Primates and Humans (00:33:23) Unlocking the Endorphin System Without Physical Touch (00:34:10) The Power of Laughter, Singing, and Storytelling in Group Bonding (00:36:00) The Limitations of Digital Interactions for Building Relationships (00:39:51) Reviving Social Clubs and Activities for Workplace Bonding (00:44:40) The Importance of Homophily in Friendships and Social Networks (00:50:40) Challenges and Solutions for Overcoming Loneliness and Building Trust (00:53:45) The Impact of Technology, Religion, and Mental Health on Social Connections (00:61:47) Reflecting on Time as a Fundamental Aspect of Social Dynamics

Robin Dunbar

Professor of Evolutionary Psychology

University of Oxford

Dr. Minqi Jiang and Dr. Marc Rigter explain an innovative new method to make the intelligence of agents more general-purpose by training them to learn many worlds before their usual goal-directed training, which we call "reinforcement learning". Their new paper is called "Reward-free curricula for training robust world models" https://arxiv.org/pdf/2306.09205.pdf https://twitter.com/MinqiJiang https://twitter.com/MarcRigter Interviewer: Dr. Tim Scarfe Please support us on Patreon, Tim is now doing MLST full-time and taking a massive financial hit. If you love MLST and want this to continue, please show your support! In return you get access to shows very early and private discord and networking. https://patreon.com/mlst We are also looking for show sponsors, please get in touch if interested mlstreettalk at gmail. MLST Discord: https://discord.gg/machine-learning-street-talk-mlst-937356144060530778

Minqi Jiang

Researcher

Meta AI

Marc Rigter

Researcher

University of Oxford

Watch behind the scenes, get early access and join the private Discord by supporting us on Patreon: https://patreon.com/mlst (public discord) https://discord.gg/aNPkGUQtc5 https://twitter.com/MLStreetTalk DOES AI HAVE AGENCY? With Professor. Karl Friston and Riddhi J. Pitliya Agency in the context of cognitive science, particularly when considering the free energy principle, extends beyond just human decision-making and autonomy. It encompasses a broader understanding of how all living systems, including non-human entities, interact with their environment to maintain their existence by minimising sensory surprise. According to the free energy principle, living organisms strive to minimize the difference between their predicted states and the actual sensory inputs they receive. This principle suggests that agency arises as a natural consequence of this process, particularly when organisms appear to plan ahead many steps in the future. Riddhi J. Pitliya is based in the computational psychopathology lab doing her Ph.D at the University of Oxford and works with Professor Karl Friston at VERSES. https://twitter.com/RiddhiJP References: THE FREE ENERGY PRINCIPLE—A PRECIS [Ramstead] https://www.dialecticalsystems.eu/contributions/the-free-energy-principle-a-precis/ Active Inference: The Free Energy Principle in Mind, Brain, and Behavior [Thomas Parr, Giovanni Pezzulo, Karl J. Friston] https://direct.mit.edu/books/oa-monograph/5299/Active-InferenceThe-Free-Energy-Principle-in-Mind The beauty of collective intelligence, explained by a developmental biologist | Michael Levin https://www.youtube.com/watch?v=U93x9AWeuOA Growing Neural Cellular Automata https://distill.pub/2020/growing-ca Carcinisation https://en.wikipedia.org/wiki/Carcinisation Prof. KENNETH STANLEY - Why Greatness Cannot Be Planned https://www.youtube.com/watch?v=lhYGXYeMq_E On Defining Artificial Intelligence [Pei Wang] https://sciendo.com/article/10.2478/jagi-2019-0002 Why? The Purpose of the Universe [Goff] https://amzn.to/4aEqpfm Umwelt https://en.wikipedia.org/wiki/Umwelt An Immense World: How Animal Senses Reveal the Hidden Realms [Yong] https://amzn.to/3tzzTb7 What's it like to be a bat [Nagal] https://www.sas.upenn.edu/~cavitch/pdf-library/Nagel_Bat.pdf COUNTERFEIT PEOPLE. DANIEL DENNETT. (SPECIAL EDITION) https://www.youtube.com/watch?v=axJtywd9Tbo We live in the infosphere [FLORIDI] https://www.youtube.com/watch?v=YLNGvvgq3eg Mark Zuckerberg: First Interview in the Metaverse | Lex Fridman Podcast #398 https://www.youtube.com/watch?v=MVYrJJNdrEg Black Mirror: Rachel, Jack and Ashley Too | Official Trailer | Netflix https://www.youtube.com/watch?v=-qIlCo9yqpY

Karl Friston

Professor / Neuroscientist

University College LondonVerses AI

Riddhi J. Pitliya

Researcher

University of Oxford

Support us! https://www.patreon.com/mlst MLST Discord: https://discord.gg/aNPkGUQtc5 Twitter: https://twitter.com/MLStreetTalk In this eye-opening discussion between Tim Scarfe and Prof. Jim Hughes, a professor of gene regulation at Oxford University, they explore the intersection of creativity, genomics, and artificial intelligence. Prof. Hughes brings his expertise in genomics and insights from his interdisciplinary research group, which includes machine learning experts, mathematicians, and molecular biologists. The conversation begins with an overview of Prof. Hughes' background and the importance of creativity in scientific research. They delve into the challenges of unlocking the secrets of the human genome and how machine learning, specifically convolutional neural networks, can assist in decoding genome function. As they discuss validation and interpretability concerns in machine learning, they acknowledge the need for experimental tests and ponder the complex nature of understanding the basic code of life. They touch upon the fascinating world of morphogenesis and emergence, considering the potential crossovers into AI and their implications for self-repairing systems in medicine. Examining the ethical and regulatory aspects of genomics and AI, the duo explores the implications of having access to someone's genome, the potential to predict traits or diseases, and the role of AI in understanding complex genetic signals. They also consider the challenges of keeping up with the rapidly expanding body of scientific research and the pressures faced by researchers in academia. To wrap up the discussion, Tim and Prof. Hughes shed light on the significance of creativity and diversity in scientific research, emphasizing the need for divergent processes and diverse perspectives to foster innovation and avoid consensus-driven convergence. Filmed at https://www.creativemachine.io/Prof. Jim Hughes: https://www.rdm.ox.ac.uk/people/jim-hughesDr. Tim Scarfe: https://xrai.glass/ Table of Contents: 1. [0:00:00] Introduction and Prof. Jim Hughes' background 2. [0:02:48] Creativity and its role in science 3. [0:07:13] Challenges in understanding the human genome 4. [0:13:20] Using convolutional neural networks to decode genome function 5. [0:15:32] Validation and interpretability concerns in machine learning 6. [0:17:56] Challenges in understanding the basic code of life 7. [0:19:36] Morphogenesis, emergence, and potential crossovers into AI 8. [0:21:38] Ethics and regulation in genomics and AI 9. [0:23:30] The role of AI in understanding and managing genetic risks 10. [0:32:37] Creativity and diversity in scientific research

Jim Hughes

Professor of Gene Regulation

University of Oxford

Brought to you by LinkedIn Jobs recruitment platform with 900M+ users, FreshBooks cloud-based small business accounting software, and Athletic Greens’s AG1 all-in-one nutritional supplement. David Deutsch (@DavidDeutschOxf) is a visiting professor of physics at the Centre for Quantum Computation, a part of the Clarendon Laboratory at Oxford University, and an honorary fellow of Wolfson College, Oxford. He works on fundamental issues in physics, particularly the quantum theory of computation and information and especially constructor theory, which he is proposing as a new way of formulating laws of nature. He is the author of The Fabric of Reality and The Beginning of Infinity, and he is an advocate of the philosophy of Karl Popper. Naval Ravikant (@naval) is the co-founder of Airchat and AngelList. He has invested in more than 100 companies, including many mega-successes, such as Twitter, Uber, Notion, Opendoor, Postmates, and Wish. You can see his latest musings on Airchat and subscribe to Naval, his podcast on wealth and happiness, on Apple Podcasts, Spotify, Overcast, or wherever you get your podcasts. You can also find his blog at nav.al. For more Naval-plus-Tim, check out my wildly popular interview with him from 2015 (nominated for “Podcast of the Year”) and our conversation from 2020. Naval also co-piloted the interviews with Ethereum creator Vitalik Buterin and famed investor Chris Dixon. Please enjoy! * This episode is brought to you by LinkedIn Jobs. Whether you are looking to hire now for a critical role or thinking about needs that you may have in the future, LinkedIn Jobs can help. LinkedIn screens candidates for the hard and soft skills you’re looking for and puts your job in front of candidates looking for job opportunities that match what you have to offer. Using LinkedIn’s active community of more than 900 million professionals worldwide, LinkedIn Jobs can help you find and hire the right person faster. When your business is ready to make that next hire, find the right person with LinkedIn Jobs. And now, you can post a job for free. Just visit LinkedIn.com/Tim. * This episode is also brought to you by FreshBooks. I’ve been talking about FreshBooks—an all-in-one invoicing + payments + accounting solution—for years now. Many entrepreneurs, as well as the contractors and freelancers that I work with, use it all the time. FreshBooks makes it super easy to track things like expenses, project time, and client info and then merge it all into great-looking invoices. And right now, there’s a special offer just for my listeners. Head over to FreshBooks.com/Tim to get 90% off your FreshBooks subscription for 4 months. * This episode is also brought to you by Athletic Greens. I get asked all the time, “If you could use only one supplement, what would it be?” My answer is usually AG1 by Athletic Greens, my all-in-one nutritional insurance. I recommended it in The 4-Hour Body in 2010 and did not get paid to do so. I do my best with nutrient-dense meals, of course, but AG further covers my bases with vitamins, minerals, and whole-food-sourced micronutrients that support gut health and the immune system. Right now, Athletic Greens is offering you their Vitamin D Liquid Formula free with your first subscription purchase—a vital nutrient for a strong immune system and strong bones. Visit AthleticGreens.com/Tim to claim this special offer today and receive the free Vitamin D Liquid Formula (and 5 free travel packs) with your first subscription purchase! That’s up to a one-year supply of Vitamin D as added value when you try their delicious and comprehensive all-in-one daily greens product. * [08:03] The impact The Fabric of Reality and The Beginning of Infinity have had on Naval. [10:07] The four strands. [13:04] Dispelling common misconceptions about science. [19:26] How does knowledge grow? [24:26] The benefits of understanding the four strands. [32:47] How quantum computing arose from trying to test a multiverse theory. [37:40] What a good explanation looks like. [42:43] How do conjecture and criticism give us a basis for optimism? [48:38] Translating knowledge into action. [51:20] Artificial intelligence (AI) vs. artificial general intelligence (AGI). [56:54] AGI is people! But how do we ensure it’ll be good people? [1:03:03] What’s taking AGI so long to get here? [1:08:59] Chemical scum that dream of distant quasars. [1:17:47] Are humans central to the universe, or just a sideshow? [1:20:17] Wealth and resources. [1:25:30] Recommended thinkers. [1:28:05] Taking Children Seriously, ToKCast, Critical Rationalists, and Popper 101. [1:31:55] David’s most interesting problems right now. [1:39:24] Parting thoughts. * For show notes and past guests on The Tim Ferriss Show, please visit tim.blog/podcast. For deals from sponsors of The Tim Ferriss Show, please visit tim.blog/podcast-sponsors Sign up for Tim’s email newsletter (5-Bullet Friday) at tim.blog/friday. For transcripts of episodes, go to tim.blog/transcripts. Discover Tim’s books: tim.blog/books. Follow Tim: Twitter: twitter.com/tferriss Instagram: instagram.com/timferriss YouTube: youtube.com/timferriss Facebook: facebook.com/timferriss LinkedIn: linkedin.com/in/timferriss Past guests on The Tim Ferriss Show include Jerry Seinfeld, Hugh Jackman, Dr. Jane Goodall, LeBron James, Kevin Hart, Doris Kearns Goodwin, Jamie Foxx, Matthew McConaughey, Esther Perel, Elizabeth Gilbert, Terry Crews, Sia, Yuval Noah Harari, Malcolm Gladwell, Madeleine Albright, Cheryl Strayed, Jim Collins, Mary Karr, Maria Popova, Sam Harris, Michael Phelps, Bob Iger, Edward Norton, Arnold Schwarzenegger, Neil Strauss, Ken Burns, Maria Sharapova, Marc Andreessen, Neil Gaiman, Neil de Grasse Tyson, Jocko Willink, Daniel Ek, Kelly Slater, Dr. Peter Attia, Seth Godin, Howard Marks, Dr. Brené Brown, Eric Schmidt, Michael Lewis, Joe Gebbia, Michael Pollan, Dr. Jordan Peterson, Vince Vaughn, Brian Koppelman, Ramit Sethi, Dax Shepard, Tony Robbins, Jim Dethmer, Dan Harris, Ray Dalio, Naval Ravikant, Vitalik Buterin, Elizabeth Lesser, Amanda Palmer, Katie Haun, Sir Richard Branson, Chuck Palahniuk, Arianna Huffington, Reid Hoffman, Bill Burr, Whitney Cummings, Rick Rubin, Dr. Vivek Murthy, Darren Aronofsky, Margaret Atwood, Mark Zuckerberg, Peter Thiel, Dr. Gabor Maté, Anne Lamott, Sarah Silverman, Dr. Andrew Huberman, and many more. See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

David Deutsch

Physicist & Author

University of Oxford

Naval Ravikant

Co-founder and Former CEO

AngelList

Support us! https://www.patreon.com/mlst MLST Discord: https://discord.gg/aNPkGUQtc5 Christopher Summerfield, Department of Experimental Psychology, University of Oxford is a Professor of Cognitive Neuroscience at the University of Oxford and a Research Scientist at Deepmind UK. His work focusses on the neural and computational mechanisms by which humans make decisions. Chris has just released an incredible new book on AI called "Natural General Intelligence". It's my favourite book on AI I have read so so far. The book explores the algorithms and architectures that are driving progress in AI research, and discusses intelligence in the language of psychology and biology, using examples and analogies to be comprehensible to a wide audience. It also tackles longstanding theoretical questions about the nature of thought and knowledge. With Chris' permission, I read out a summarised version of Chapter 2 from his book on which was on Intelligence during the 30 minute MLST introduction. Buy his book here: https://global.oup.com/academic/product/natural-general-intelligence-9780192843883?cc=gb&lang=en& YT version: https://youtu.be/31VRbxAl3t0 Interviewer: Dr. Tim Scarfe TOC: [00:00:00] Walk and talk with Chris on Knowledge and Abstractions [00:04:08] Intro to Chris and his book [00:05:55] (Intro) Tim reads Chapter 2: Intelligence [00:09:28] Intro continued: Goodhart's law [00:15:37] Intro continued: The "swiss cheese" situation [00:20:23] Intro continued: On Human Knowledge [00:23:37] Intro continued: Neats and Scruffies [00:30:22] Interview kick off [00:31:59] What does it mean to understand? [00:36:18] Aligning our language models [00:40:17] Creativity [00:41:40] "Meta" AI and basins of attraction [00:51:23] What can Neuroscience impart to AI [00:54:43] Sutton, neats and scruffies and human alignment [01:02:05] Reward is enough [01:19:46] Jon Von Neumann and Intelligence [01:23:56] Compositionality References: The Language Game (Morten H. Christiansen, Nick Chater https://www.penguin.co.uk/books/441689/the-language-game-by-morten-h-christiansen-and--nick-chater/9781787633483 Theory of general factor (Spearman) https://www.proquest.com/openview/7c2c7dd23910c89e1fc401e8bb37c3d0/1?pq-origsite=gscholar&cbl=1818401 Intelligence Reframed (Howard Gardner) https://books.google.co.uk/books?hl=en&lr=&id=Qkw4DgAAQBAJ&oi=fnd&pg=PT6&dq=howard+gardner+multiple+intelligences&ots=ERUU0u5Usq&sig=XqiDgNUIkb3K9XBq0vNbFmXWKFs#v=onepage&q=howard%20gardner%20multiple%20intelligences&f=false The master algorithm (Pedro Domingos) https://www.amazon.co.uk/Master-Algorithm-Ultimate-Learning-Machine/dp/0241004543 A Thousand Brains: A New Theory of Intelligence (Jeff Hawkins) https://www.amazon.co.uk/Thousand-Brains-New-Theory-Intelligence/dp/1541675819 The bitter lesson (Rich Sutton) http://www.incompleteideas.net/IncIdeas/BitterLesson.html

Chris Summerfield

Professor of Cognitive Neuroscience

University of Oxford

YT version (with references): https://www.youtube.com/watch?v=lxaTinmKxs0 Support us! https://www.patreon.com/mlst MLST Discord: https://discord.gg/aNPkGUQtc5 Carla Cremer and Igor Krawczuk argue that AI risk should be understood as an old problem of politics, power and control with known solutions, and that threat models should be driven by empirical work. The interaction between FTX and the Effective Altruism community has sparked a lot of discussion about the dangers of optimization, and Carla's Vox article highlights the need for an institutional turn when taking on a responsibility like risk management for humanity. Carla's “Democratizing Risk” paper found that certain types of risks fall through the cracks if they are just categorized into climate change or biological risks. Deliberative democracy has been found to be a better way to make decisions, and AI tools can be used to scale this type of democracy and be used for good, but the transparency of these algorithms to the citizens using the platform must be taken into consideration. Aggregating people’s diverse ways of thinking about a problem and creating a risk-averse procedure gives a likely, highly probable outcome for having converged on the best policy. There needs to be a good reason to trust one organization with the risk management of humanity and all the different ways of thinking about risk must be taken into account. AI tools can help to scale this type of deliberative democracy, but the transparency of these algorithms must be taken into consideration. The ambition of the EA community and Altruism Inc. is to protect and do risk management for the whole of humanity and this requires an institutional turn in order to do it effectively. The dangers of optimization are real, and it is essential to ensure that the risk management of humanity is done properly and ethically. By understanding the importance of aggregating people’s diverse ways of thinking about a problem, and creating a risk-averse procedure, it is possible to create a likely, highly probable outcome for having converged on the best policy. Carla Zoe Cremer https://carlacremer.github.io/ Igor Krawczuk https://krawczuk.eu/ Interviewer: Dr. Tim Scarfe TOC: [00:00:00] Introduction: Vox article and effective altruism / FTX [00:11:12] Luciano Floridi on Governance and Risk [00:15:50] Connor Leahy on alignment [00:21:08] Ethan Caballero on scaling [00:23:23] Alignment, Values and politics [00:30:50] Singularitarians vs AI-thiests [00:41:56] Consequentialism [00:46:44] Does scale make a difference? [00:51:53] Carla's Democratising risk paper [01:04:03] Vox article - How effective altruists ignored risk [01:20:18] Does diversity breed complexity? [01:29:50] Collective rationality [01:35:16] Closing statements

Carla Cremer

Researcher

University of Oxford

Igor Krawczuk

Researcher

EPFL

Will MacAskill is one of the founders of the Effective Altruist movement and the author of the upcoming book, What We Owe The Future. We talk about improving the future, risk of extinction & collapse, technological & moral change, problems of academia, who changes history, and much more. Watch on YouTube. Listen on Apple Podcasts, Spotify, or any other podcast platform. Episode website + Transcript here. Follow Will on Twitter. Follow me on Twitter for updates on future episodes. Subscribe to find out about future episodes! Timestamps (00:23) - Effective Altruism and Western values (07:47) - The contingency of technology (12:02) - Who changes history? (18:00) - Longtermist institutional reform (25:56) - Are companies longtermist? (28:57) - Living in an era of plasticity (34:52) - How good can the future be? (39:18) - Contra Tyler Cowen on what’s most important (45:36) - AI and the centralization of power (51:34) - The problems with academia Please share if you enjoyed this episode! Helps out a ton! Transcript Dwarkesh Patel 0:06 Okay, today I have the pleasure of interviewing William MacAskill. Will is one of the founders of the Effective Altruism movement, and most recently, the author of the upcoming book, What We Owe The Future. Will, thanks for coming on the podcast. Will MacAskill 0:20 Thanks so much for having me on. Effective Altruism and Western values Dwarkesh Patel 0:23 My first question is: What is the high-level explanation for the success of the Effective Altruism movement? Is it itself an example of the contingencies you talk about in the book? Will MacAskill 0:32 Yeah, I think it is contingent. Maybe not on the order of, “this would never have happened,” but at least on the order of decades. Evidence that Effective Altruism is somewhat contingent is that similar ideas have been promoted many times during history, and not taken on. We can go back to ancient China, the Mohists defended an impartial view of morality, and took very strategic actions to help all people. In particular, providing defensive assistance to cities under siege. Then, there were early utilitarians. Effective Altruism is broader than utilitarianism, but has some similarities. Even Peter Singer in the 70s had been promoting the idea that we should be giving most of our income to help the very poor — and didn’t get a lot of traction until early 2010 after GiveWell and Giving What We Can launched. What explains the rise of it? I think it was a good idea waiting to happen. At some point, the internet helped to gather together a lot of like-minded people which wasn’t possible otherwise. There were some particularly lucky events like Alex meeting Holden and me meeting Toby that helped catalyze it at the particular time it did. Dwarkesh Patel 1:49 If it's true, as you say, in the book, that moral values are very contingent, then shouldn't that make us suspect that modern Western values aren't that good? They're mediocre, or worse, because ex ante, you would expect to end up with a median of all the values we could have had at this point. Obviously, we'd be biased in favor of whatever values we were brought up in. Will MacAskill 2:09 Absolutely. Taking history seriously and appreciating the contingency of values, appreciating that if the Nazis had won the World War, we would all be thinking, “wow, I'm so glad that moral progress happened the way it did, and we don't have Jewish people around anymore. What huge moral progress we had then!” That's a terrifying thought. I think it should make us take seriously the fact that we're very far away from the moral truth. One of the lessons I draw in the book is that we should not think we're at the end of moral progress. We should not think, “Oh, we should lock in the Western values we have.” Instead, we should spend a lot of time trying to figure out what's actually morally right, so that the future is guided by the right values, rather than whichever happened to win out. Dwarkesh Patel 2:56 So that makes a lot of sense. But I'm asking a slightly separate question—not only are there possible values that could be better than ours, but should we expect our values - we have the sense that we've made moral progress (things are better than they were before or better than most possible other worlds in 2100 or 2200)- should we not expect that to be the case? Should our priors be that these are ‘meh’ values? Will MacAskill 3:19 Our priors should be that our values are as good as expected on average. Then you can make an assessment like, “Are other values of today going particularly well?” There are some arguments you could make for saying no. Perhaps if the Industrial Revolution happened in India, rather than in Western Europe, then perhaps we wouldn't have wide-scale factory farming—which I think is a moral atrocity. Having said that, my view is to think that we're doing better than average. If civilization were just a redraw, then things would look worse in terms of our moral beliefs and attitudes. The abolition of slavery, the feminist movement, liberalism itself, democracy—these are all things that we could have lost and are huge gains. Dwarkesh Patel 4:14 If that's true, does that make the prospect of a long reflection dangerous? If moral progress is a random walk, and we've ended up with a lucky lottery, then you're possibly reversing. Maybe you're risking regression to the mean if you just have 1,000 years of progress. Will MacAskill 4:30 Moral progress isn't a random walk in general. There are many forces that act on culture and on what people believe. One of them is, “What’s right, morally speaking? What's their best arguments support?” I think it's a weak force, unfortunately. The idea of lumbar flexion is getting society into a state that before we take any drastic actions that might lock in a particular set of values, we allow this force of reason and empathy and debate and goodhearted model inquiry to guide which values we end up with. Are we unwise? Dwarkesh Patel 5:05 In the book, you make this interesting analogy where humans at this point in history are like teenagers. But another common impression that people have of teenagers is that they disregard wisdom and tradition and the opinions of adults too early and too often. And so, do you think it makes sense to extend the analogy this way, and suggest that we should be Burkean Longtermists and reject these inside-view esoteric threats? Will MacAskill 5:32 My view goes the opposite of the Burkean view. We are cultural creatures in our nature, and are very inclined to agree with what other people think even if we don't understand the underlying mechanisms. It works well in a low-change environment. The environment we evolved towards didn't change very much. We were hunter-gatherers for hundreds of years. Now, we're in this period of enormous change, where the economy is doubling every 20 years, new technologies arrive every single year. That's unprecedented. It means that we should be trying to figure things out from first principles. Dwarkesh Patel 6:34 But at current margins, do you think that's still the case? If a lot of EA and longtermist thought is first principles, do you think that more history would be better than the marginal first-principles thinker? Will MacAskill 6:47 Two things. If it's about an understanding of history, then I'd love EA to have a better historical understanding. The most important subject if you want to do good in the world is philosophy of economics. But we've got that in abundance compared to there being very little historical knowledge in the EA community. Should there be even more first-principles thinking? First-principles thinking paid off pretty well in the course of the Coronavirus pandemic. From January 2020, my Facebook wall was completely saturated with people freaking out, or taking it very seriously in a way that the existing institutions weren't. The existing institutions weren't properly updating to a new environment and new evidence. The contingency of technology Dwarkesh Patel 7:47 In your book, you point out several examples of societies that went through hardship. Hiroshima after the bombings, Europe after the Black Death—they seem to have rebounded relatively quickly. Does this make you think that perhaps the role of contingency in history, especially economic history is not that large? And it implies a Solow model of growth? That even if bad things happen, you can rebound and it really didn't matter? Will MacAskill 8:17 In economic terms, that's the big difference between economic or technological progress and moral progress. In the long run, economic or technological progress is very non-contingent. The Egyptians had an early version of the steam engine, semaphore was only developed very late yet could have been invented thousands of years in the past. But in the long run, the instrumental benefits of tech progress, and the incentives towards tech progress and economic growth are so strong, that we get there in a wide array of circumstances. Imagine there're thousands of different societies, and none are growing except for one. In the long run, that one becomes the whole economy. Dwarkesh Patel 9:10 It seems that particular example you gave of the Egyptians having some ancient form of a steam engine points towards there being more contingency? Perhaps because the steam engine comes up in many societies, but it only gets turned into an industrial revolution in one? Will MacAskill 9:22 In that particular case, there's a big debate about whether quality of metalwork made it actually possible to build a proper steam engine at that time. I mentioned those to share some amazing examples of contingency prior to the Industrial Revolution. It's still contingency on the order of centuries to thousands of years. Post industrial-revolution world, there's much less contingency. It's much harder to see technologies that wouldn't have happened within decades if they hadn't been developed when they were. Dwarkesh Patel 9:57 The model here is, “These general-purpose changes in the state of technology are contingent, and it'd be very important to try to engineer one of those. But other than that, it's going to get done by some guy creating a start-up anyways?” Will MacAskill 10:11 Even in the case of the steam engine that seemed contingent, it gets developed in the long run. If the Industrial Revolution hadn't happened in Britain in the 18th century, would it have happened at some point? Would similar technologies that were vital to the industrial revolution developed? Yes, there are very strong incentives for doing so. If there’s a culture that's into making textiles in an automated way as opposed to England in the 18th century, then that economy will take over the world. There's a structural reason why economic growth is much less contingent than moral progress. Dwarkesh Patel 11:06 When people think of somebody like Norman Borlaug and the Green Revolution. It's like, “If you could have done something that, you'd be the greatest person in the 20th century.” Obviously, he's still a very good man, but would that not be our view? Do you think the green revolution would have happened anyways? Will MacAskill 11:22 Yes. Norman Borlaug is sometimes credited with saving a billion lives. He was huge. He was a good force for the world. Had Norman Borlaug not existed, I don’t think a billion people would have died. Rather, similar developments would have happened shortly afterwards. Perhaps he saved tens of millions of lives—and that's a lot of lives for a person to save. But, it's not as many as simply saying, “Oh, this tech was used by a billion people who would have otherwise been at risk of starvation.” In fact, not long afterwards, there were similar kinds of agricultural development. Who changes history? Dwarkesh Patel 12:02 What kind of profession or career choice tends to lead to the highest counterfactual impact? Is it moral philosophers? Will MacAskill 12:12 Not quite moral philosophers, although there are some examples. Sticking on science technology, if you look at Einstein, theory of special relativity would have been developed shortly afterwards. However, theory of general relativity was plausibly decades in advance. Sometimes, you get surprising leaps. But, we're still only talking about decades rather than millennia. Moral philosophers could make long-term difference. Marx and Engels made an enormous, long-run difference. Religious leaders like Mohammed, Jesus, and Confucius made enormous and contingent, long-run difference. Moral activists as well. Dwarkesh Patel 13:04 If you think that the changeover in the landscape of ideas is very quick today, would you still think that somebody like Marx will be considered very influential in the long future? Communism lasted less than a century, right? Will MacAskill 13:20 As things turned out, Marx will not be influential over the long term future. But that could have gone another way. It's not such a wildly different history. Rather than liberalism emerging dominant in the 20th century, it was communism. The better technology gets, the better the ruling ideology is to cement its ideology and persist for a long time. You can get a set of knock-on effects where communism wins the war of ideas in the 20th century. Let’s say a world-government is based around those ideas, then, via anti-aging technology, genetic-enhancement technology, cloning, or artificial intelligence, it's able to build a society that possesses forever in accordance with that ideology. Dwarkesh Patel 14:20 The death of dictators is especially interesting when you're thinking about contingency because there are huge changes in the regime. It makes me think the actual individual there was very important and who they happened to be was contingent and persistent in some interesting ways. Will MacAskill 14:37 If you've got a dictatorship, then you've got single person ruling the society. That means it's heavily contingent on the views, values, beliefs, and personality of that person. Scientific talent Dwarkesh Patel 14:48 Going back to the second nation, in the book, you're very concerned about fertility. It seems your model about scientific and technological progress happens is number of people times average researcher productivity. If resource productivity is declining and the number of people isn't growing that fast, then that's concerning. Will MacAskill 15:07 Yes, number of people times fraction of the population devoted to R&D. Dwarkesh Patel 15:11 Thanks for the clarification. It seems that there have been a lot of intense concentrations of talent and progress in history. Venice, Athens, or even something like FTX, right? There are 20 developers making this a multibillion dollar company—do these examples suggest that organization and congregation of researchers matter more than the total amount? Will MacAskill 15:36 The model works reasonably well. Throughout history, you start from a very low technological baseline compared to today. Most people aren't even trying to innovate. One argument for why Baghdad lost its Scientific Golden Age is because the political landscape changed such that what was incentivized was theological investigation rather than scientific investigation in the 10th/11th century AD. Similarly, one argument for why Britain had a scientific and industrial revolution rather than Germany was because all of the intellectual talent in Germany was focused on making amazing music. That doesn't compound in the way that making textiles does. If you look at like Sparta versus Athens, what was the difference? They had different cultures and intellectual inquiry was more rewarded in Athens. Because they're starting from a lower base, people trying to do something that looks like what we now think of as intellectual inquiry have an enormous impact. Dwarkesh Patel 16:58 If you take an example like Bell Labs, the low-hanging fruit is gone by the late 20th century. You have this one small organization that has six Nobel Prizes. Is this a coincidence? Will MacAskill 17:14 I wouldn't say that at all. The model we’re working with is the size of the population times the fraction of the population doing R&D. It's the simplest model you can have. Bell Labs is punching above its weight. You can create amazing things from a certain environment with the most productive people and putting them in an environment where they're ten times more productive than they would otherwise be. However, when you're looking at the grand sweep of history, those effects are comparatively small compared to the broader culture of a society or the sheer size of a population. Longtermist institutional reform Dwarkesh Patel 18:00 I want to talk about your paper on longtermist institutional reform. One of the things you advocate in this paper is that we should have one of the houses be dedicated towards longtermist priorities. Can you name some specific performance metrics you would use to judge or incentivize the group of people who make up this body? Will MacAskill 18:23 The thing I'll caveat with longtermist institutions is that I’m pessimistic about them. If you're trying to represent or even give consideration to future people, you have to face the fact that they're not around and they can't lobby for themselves. However, you could have an assembly of people who have some legal regulatory power. How would you constitute that? My best guess is you have a random selection from the population? How would you ensure that incentives are aligned? In 30-years time, their performance will be assessed by a panel of people who look back and assess the policies’ effectiveness. Perhaps the people who are part of this assembly have their pensions paid on the basis of that assessment. Secondly, the people in 30-years time, both their policies and their assessment of the previous 30-years previous assembly get assessed by another assembly, 30-years after that, and so on. Can you get that to work? Maybe in theory—I’m skeptical in practice, but I would love some country to try it and see what happens. There is some evidence that you can get people to take the interests of future generations more seriously by just telling them their role. There was one study that got people to put on ceremonial robes, and act as trustees of the future. And they did make different policy recommendations than when they were just acting on the basis of their own beliefs and self-interest. Dwarkesh Patel 20:30 If you are on that board that is judging these people, is there a metric like GDP growth that would be good heuristics for assessing past policy decisions? Will MacAskill 20:48 There are some things you could do: GDP growth, homelessness, technological progress. I would absolutely want there to be an expert assessment of the risk of catastrophe. We don't have this yet, but imagine a panel of super forecasters predicting the chance of a war between great powers occurring in the next ten years that gets aggregated into a war index. That would be a lot more important than the stock market index. Risk of catastrophe would be helpful to feed into because you wouldn't want something only incentivizing economic growth at the expense of tail risks. Dwarkesh Patel 21:42 Would that be your objection to a scheme like Robin Hanson’s about maximizing the expected future GDP using prediction markets and making decisions that way? Will MacAskill 21:50 Maximizing future GDP is an idea I associate with Tyler Cowen. With Robin Hanson’s idea of voting on values but betting on beliefs, if people can vote on what collection of goods they want, GDP and unemployment might be good metrics. Beyond that, it's pure prediction markets. It's something I'd love to see tried. It’s an idea of speculative political philosophy about how a society could be extraordinarily different in structure that is incredibly neglected. Do I think it'll work in practice? Probably not. Most of these ideas wouldn't work. Prediction markets can be gamed or are simply not liquid enough. There hasn’t been a lot of success in prediction markets compared to forecasting. Perhaps you can solve these things. You have laws about what things can be voted on or predicted in the prediction market, you could have government subsidies to ensure there's enough liquidity. Overall, it's likely promising and I'd love to see it tried out on a city-level or something. Dwarkesh Patel 23:13 Let’s take a scenario where the government starts taking the impact on the long-term seriously and institutes some reforms to integrate that perspective. As an example, you can take a look at the environmental movement. There're environmental review boards that will try to assess the environmental impact of new projects and repeal any proposals based on certain metrics. The impact here, at least in some cases, has been that groups that have no strong, plausible interest in the environment are able to game these mechanisms in order to prevent projects that would actually help the environment. With longtermism, it takes a long time to assess the actual impact of something, but policymakers are tasked with evaluating the long term impacts of something. Are you worried that it'd be a system that'd be easy to game by malicious actors? And they'd ask, “What do you think went wrong with the way that environmentalism was codified into law?” Will MacAskill 24:09 It's potentially a devastating worry. You create something to represent future people, but they're not allowed to lobby themselves (it can just be co-opted). My understanding of environmental impact statements has been similar. Similarly, it's not like the environment can represent itself—it can't say what its interests are. What is the right answer there? Maybe there are speculative proposals about having a representative body that assesses these things and elect jobs by people in 30-years time. That's the best we've got at the moment, but we need a lot more thought to see if any of these proposals would be robust for the long term rather than things that are narrowly-focused. Regulation to have liability insurance for dangerous bio labs is not about trying to represent the interests of future generations. But, it's very good for the long-term. At the moment, if longtermists are trying to change the government, let's focus on a narrow set of institutional changes that are very good for the long-term even if they're not in the game of representing the future. That's not to say I'm opposed to all such things. But, there are major problems with implementation for any of them. Dwarkesh Patel 25:35 If we don't know how we would do it correctly, did you have an idea of how environmentalism could have been codified better? Why was that not a success in some cases? Will MacAskill 25:46 Honestly, I don't have a good understanding of that. I don't know if it's intrinsic to the matter or if you could’ve had some system that wouldn't have been co-opted in the long-term. Are companies longtermist? Dwarkesh Patel 25:56 Theoretically, the incentives of our most long-term U.S. institutions is to maximize future cash flow. Explicitly and theoretically, they should have an incentive to do the most good they can for their own company—which implies that the company can’t be around if there’s an existential risk… Will MacAskill 26:18 I don't think so. Different institutions have different rates of decay associated with them. So, a corporation that is in the top 200 biggest companies has a half-life of only ten years. It’s surprisingly short-lived. Whereas, if you look at universities Oxford and Cambridge are 800 years old. University of Bologna is even older. These are very long-lived institutions. For example, Corpus Christi at Oxford was making a decision about having a new tradition that would occur only every 400 years. It makes that kind of decision because it is such a long-lived institution. Similarly, the legends can be even longer-lived again. That type of natural half-life really affects the decisions a company would make versus a university versus a religious institution. Dwarkesh Patel 27:16 Does that suggest that there's something fragile and dangerous about trying to make your institution last for a long time—if companies try to do that and are not able to? Will MacAskill 27:24 Companies are composed of people. Is it in the interest of a company to last for a long time? Is it in the interests of the people who constitute the company (like the CEO and the board and the shareholders) for that company to last a long time? No, they don't particularly care. Some of them do, but most don't. Whereas other institutions go both ways. This is the issue of lock-in that I talked about at length in What We Owe The future: you get moments of plasticity during the formation of a new institution. Whether that’s the Christian church or the Constitution of the United States, you lock-in a certain set of norms. That can be really good. Looking back, the U.S. Constitution seems miraculous as the first democratic constitution. As I understand it, it was created over a period of four months seems to have stood the test of time. Alternatively, lock-in norms could be extremely dangerous. There were horrible things in the U.S. Constitution like the legal right to slavery proposed as a constitutional amendment. If that had locked in, it would have been horrible. It's hard to answer in the abstract because it depends on the thing that's persisting for a long time. Living in an era of plasticity Dwarkesh Patel 28:57 You say in the book that you expect our current era to be a moment of plasticity. Why do you think that is? Will MacAskill 29:04 There are specific types of ‘moments of plasticity’ for two reasons. One is a world completely unified in a way that's historically unusual. You can communicate with anyone instantaneously and there's a great diversity of moral views. We can have arguments, like people coming on your podcast can debate what's morally correct. It's plausible to me that one of many different sets of moral views become the most popular ultimately. Secondly, we're at this period where things can really change. But, it's a moment of plasticity because it could plausibly come to an end — and the moral change that we're used to could end in the coming decades. If there was a single global culture or world government that preferred ideological conformity, combined with technology, it becomes unclear why that would end over the long-term? The key technology here is Artificial Intelligence. The point in time (which may be sooner than we think) where the rulers of the world are digital rather than biological, that [ideological conformity] could persist. Once you've got that and a global hegemony of a single ideology, there's not much reason for that set of values to change over time. You've got immortal leaders and no competition. What are the other kind of sources of value-change over time? I think they can be accounted for too. Dwarkesh Patel 30:46 Isn't the fact that we are in a time of interconnectedness that won't last if we settle space — isn't that bit of reason for thinking that lock-in is not especially likely? If your overlords are millions of light years away, how well can they control you? Will MacAskill 31:01 The “whether” you have is whether the control will happen before the point of space settlement. If we took to space one day, and there're many different settlements and different solar systems pursuing different visions of the good, then you're going to maintain diversity for a very long time (given the physics of the matter). Once a solar system has been settled, it's very hard for other civilizations to come along and conquer you—at least if we're at a period of technological maturity where there aren't groundbreaking technologies to be discovered. But, I'm worried that the control will happen earlier. I'm worried the control might happen this century, within our lifetimes. I don't think it’s very likely, but it's seriously on the table - 10% or something? Dwarkesh Patel 31:53 Hm, right. Going back to the long-term of the longtermism movement, there are many instructive foundations that were set up about a century ago like the Rockefeller Foundation, Carnegie Foundation. But, they don't seem to be especially creative or impactful today. What do you think went wrong? Why was there, if not value drift, some decay of competence and leadership and insight? Will MacAskill 32:18 I don't have strong views about those particular examples, but I have two natural thoughts. For organizations that want to persist a long time and keep having an influence for a long time, they’ve historically specified their goals in far too narrow terms. One fun example is Benjamin Franklin. He invested a thousand pounds for each of the cities of Philadelphia and Boston to pay out after 100 years and then 200 years for different fractions of the amount invested. But, he specified it to help blacksmith apprentices. You might think this doesn't make much sense when you’re in the year 2000. He could have invested more generally: for the prosperity of people in Philadelphia and Boston. It would have had plausibly more impact. The second is a ‘regression to the mean’ argument. You have some new foundation and it's doing an extraordinary amount of good as the Rockefeller Foundation did. Over time, if it's exceptional in some dimension, it's probably going to get closer to average on that dimension. This is because you’re changing the people involved. If you've picked exceptionally competent and farsighted people, the next generation are statistically going to be less so. Dwarkesh Patel 33:40 Going back to that hand problem: if you specify your mission too narrowly and it doesn't make sense in the future—is there a trade off? If you're too broad, you make space for future actors—malicious or uncreative—to take the movement in ways that you would not approve of? With regards to doing good for Philadelphia, what if it turns into something that Ben Franklin would not have thought is good for Philadelphia? Will MacAskill 34:11 It depends on what your values and views are. If Benjamin Franklin only cared about blacksmith's apprentices, then he was correct to specify it. But my own values tend to be quite a bit more broad than that. Secondly, I expect people in the future to be smarter and more capable. It’s certainly the trend over time. In which case, if we’re sharing similar broad goals, and they're implementing it in a different way, then they have it. How good can the future be? Dwarkesh Patel 34:52 Let's talk about how good we should expect the future to be. Have you come across Robin Hanson’s argument that we’ll end up being subsistence-level ems because there'll be a lot of competition and minimizing compute per digital person will create a barely-worth-living experience for every entity? Will MacAskill 35:11 Yeah, I'm familiar with the argument. But, we should distinguish the idea that ems are at subsistence level from the idea that we would have bad lives. So subsistence means that you get a balance of income per capita and population growth such that being poorer would cause deaths to outweigh additional births. That doesn't tell you about their well-being. You could be very poor as an emulated being but be in bliss all the time. That's perfectly consistent with the Malthusian theory. It might seem far away from the best possible future, but it could still be very good. At subsistence, those ems could still have lives that are thousands of times better than ours. Dwarkesh Patel 36:02 Speaking of being poor and happy, there was a very interesting section in the chapter where you mentioned the study you had commissioned: you were trying to find out if people in the developing world find life worth living. It turns out that 19% of Indians would not want to relive their life every moment. But, 31% of Americans said that they would not want to relive their life at every moment? So, why are Indians seemingly much happier at less than a tenth of the GDP per capita? Will MacAskill 36:29 I think the numbers are lower than that from memory, at least. From memory, it’s something more like 9% of Indians wouldn't want to live their lives again if they had the option, and 13% of Americans said they wouldn’t. You are right on the happiness metric, though. The Indians we surveyed were more optimistic about their lives, happier with their lives than people in the US were. Honestly, I don't want to generalize too far from that because we were sampling comparatively poor Americans to comparatively well-off Indians. Perhaps it's just a sample effect. There are also weird interactions with Hinduism and the belief in reincarnation that could mess up the generalizability of this. On one hand, I don't want to draw any strong conclusion from that. But, it is pretty striking as a piece of information, given that you find people's well-being in richer countries considerably happier than poorer countries, on average. Dwarkesh Patel 37:41 I guess you do generalize in a sense that you use it as evidence that most lives today are living, right? Will MacAskill 37:50 Exactly. So, I put together various bits of evidence, where approximately 10% of people in the United States and 10% of people in India seem to think that their lives are net negative. They think they contain more suffering than happiness and wouldn't want to be reborn and live the same life if they could. There's another scripture study that looks at people in United States/other wealthy countries, and asks them how much of their conscious life they'd want to skip if they could. Skipping here means that blinking would reach you to the end of whatever activity you're engaging with. For example, perhaps I hate this podcast so much that I would rather be unconscious than be talking to you. In which case, I'd have the option of skipping, and it would be over after 30 minutes. If you look at that, and then also asked people about the trade offs they would be willing to make as a measure of intensity of how much they're enjoying a certain experience, you reach the conclusion that a little over 10% of people regarded their life that day as being surveyed worse than if they'd been unconscious the entire day. Contra Tyler Cowen on what’s most important Dwarkesh Patel 39:18 Jumping topics here a little bit, on the 80,000 Hours Podcast, you said that you expect scientists who are explicitly trying to maximize their impact might have an adverse impact because they might be ignoring the foundational research that wouldn't be obvious in this way of thinking, but might be more important. Do you think this could be a general problem with longtermism? If you were trying to find the most important things that are important long-term, you might be missing things that wouldn't be obvious thinking this way? Will MacAskill 39:48 Yeah, I think that's a risk. Among the ways that people could argue against my general set of views, I argue that we should be doing fairly specific and targeted things like trying to make AI safe, well-govern the rise of AI, reduce worst-case pandemics that can kill us all, prevent a Third World War, ensure that good values are promoted, and avoid value lock-in. But, some people could argue (and people like Tyler Cowen and Patrick Collison do), that it's very hard to predict the future impact of your actions. It's a mug's game to even try. Instead, you should look at the things that have done loads of good consistently in the past, and try to do the same things. In particular, they might argue that means technological progress or boosting economic growth. I dispute that. It's not something I can give a completely knock-down argument to because we don’t know when we will find out who's right. Maybe in thousand-years time. But one piece of evidence is the success of forecasters in general. This also was true for Tyler Cowen, but people in Effective Altruism were realizing that the Coronavirus pandemic was going to be a big deal for them. At an early stage, they were worrying about pandemics far in advance. There are some things that are actually quite predictable. For example, Moore's Law has held up for over 70 years. The idea that AI systems are gonna get much larger and leading models are going to get more powerful are on trend. Similarly, the idea that we will be soon be able to develop viruses of unprecedented destructive power doesn’t feel too controversial. Even though it’s hard to predict loads of things, there are going to be tons of surprises. There are some things, especially when it comes to fairly long-standing technological trends, that we can make reasonable predictions — at least about the range of possibilities that are on the table. Dwarkesh Patel 42:19 It sounds like you're saying that the things we know are important now. But, if something didn't turn out, a thousand years ago, looking back to be very important, it wouldn't be salient to us now? Will MacAskill 42:31 What I was saying with me versus Patrick Collison and Tyler Cowen, who is correct? We will only get that information in a thousand-years time because we're talking about impactful strategies for the long-term. We might get suggestive evidence earlier. If me and others engaging in longtermism are making specific, measurable forecasts about what is going to happen with AI, or advances in biotechnology, and then are able to take action such that we are clearly reducing certain risks, that's pretty good evidence in favor of our strategy. Whereas, they're doing all sorts of stuff, but not make firm predictions about what's going to happen, but then things pop out of that that are good for the long-term (say we measure this in ten-years time), that would be good evidence for their view. Dwarkesh Patel 43:38 You were saying earlier about the contingency in technology implies that given their worldview, even if you're trying to maximize what in the past is at the most impact, if what's had the most impact in the past is changing values, then economic growth might be the most important thing? Or trying to change the rate of economic growth? Will MacAskill 43:57 I really do take the argument seriously of how people have acted in the past, especially for people trying to make a long-lasting impact. What things that they do that made sense and whatnot. So, towards the end of the 19th century, John Stuart Mill and the other early utilitarians had this longtermist wave where they started taking the interests of future generations very seriously. Their main concern was Britain running out of coal, and therefore, future generations would be impoverished. It's pretty striking because they had a very bad understanding of how the economy works. They hadn't predicted that we would be able to transition away from coal with continued innovation. Secondly, they had enormously wrong views about how much coal and fossil fuels there were in the world. So, that particular action didn't make any sense given what we know now. In fact, that particular action of trying to keep coal in the ground, given Britain at the time where we're talking about much lower amounts of coal—so small that the climate change effect is negligible at that level—probably would have been harmful. But, we could look at other things that John Stuart Mill could have done such promoting better values. He campaigned for women's suffrage. He was the first British MP. In fact, even the first politician in the world to promote women's suffrage - that seems to be pretty good. That seems to have stood the test of time. That's one historical data point. But potentially, we can learn a more general lesson there. AI and the centralization of power Dwarkesh Patel 45:36 Do you think the ability of your global policymakers to come to a consensus is on net, a good or a bad thing? On the positive, maybe it helps around some dangerous tech from taking off, but on the negative side, prevent human challenge trials that cause some lock-in in the future. On net, what do you think about that trend? Will MacAskill 45:54 The question of global integration, you're absolutely right, it's double-sided. One hand, it can help us reduce global catastrophic risks. The fact that the world was able to come come together and ban Chlorofluorocarbons was one of the great events of the last 50 years, allowing the hole in the ozone layer to to repair itself. But on the other hand, if it means we all converge to one monoculture and lose out on diversity, that's potentially bad. We could lose out on the most possible value that way. The solution is doing the good bits and not having the bad bits. For example, in a liberal constitution, you can have a country that is bound in certain ways by its constitution and by certain laws yet still enables a flourishing diversity of moral thought and different ways of life. Similarly, in the world, you can have very strong regulation and treaties that only deal with certain global public goods like mitigation of climate change, prevention of development of the next generation of weapons of mass destruction without having some very strong-arm global government that implements a particular vision of the world. Which way are we going at the moment? It seems to me we've been going in a pretty good and not too worrying direction. But, that could change. Dwarkesh Patel 47:34 Yeah, it seems the historical trend is when you have a federated political body that even if constitutionally, the Central Powers constrain over time, they tend to gain more power. You can look at the U.S., you can look at the European Union. But yeah, that seems to be the trend. Will MacAskill 47:52 Depending on the culture that's embodied there, it's potentially a worry. It might not be if the culture itself is liberal and promoting of moral diversity and moral change and moral progress. But, that needn't be the case. Dwarkesh Patel 48:06 Your theory of moral change implies that after a small group starts advocating for a specific idea, it may take a century or more before that idea reaches common purchase. To the extent that you think this is a very important century (I know you have disagreements about that with with others), does that mean that there isn't enough time for longtermism to gain by changing moral values? Will MacAskill 48:32 There are lots of people I know and respect fairly well who think that Artificial General Intelligence will likely lead to singularity-level technological progress and extremely rapid rate of technological progress within the next 10-20 years. If so, you’re right. Value changes are something that pay off slowly over time. I talk about moral change taking centuries historically, but it can be much faster today. The growth of the Effective Altruism movement is something I know well. If that's growing at something like 30% per year, compound returns mean that it's not that long. That's not growth. That's not change that happens on the order of centuries. If you look at other moral movements like gay rights movement, very fast moral change by historical standards. If you're thinking that we've got ten years till the end of history, then don't broadly try and promote better values. But, we should have a very significant probability mass on the idea that we will not hit some historical end of this century. In those worlds, promoting better values could pay off like very well. Dwarkesh Patel 49:59 Have you heard of Slime Mold Time Mold Potato Diet? Will MacAskill 50:03 I have indeed heard of Slime Mold Time Mold Potato Diet, and I was tempted as a gimmick to try it. As I'm sure you know, potato is close to a superfood, and you could survive indefinitely on butter mashed potatoes if you occasionally supplement with something like lentils and oats. Dwarkesh Patel 50:25 Hm, interesting. Question about your career: why are you still a professor? Does it still allow you to the things that you would otherwise have been doing like converting more SBF’s and making moral philosophy arguments for EA? Curious about that. Will MacAskill 50:41 It's fairly open to me what I should do, but I do spend significant amounts of time co-founding organizations or being on the board of those organizations I've helped to set up. More recently, working closely with the Future Fund, SBF’s new foundation, and helping them do as much good as possible. That being said, if there's a single best guess for what I want to do longer term, and certainly something that plays to my strengths better, it's developing ideas, trying to get the big picture roughly right, and then communicating them in a way that's understandable and gets more people to get off their seats and start to do a lot of good for the long-term. I’ve had a lot of impact that way. From that perspective, having an Oxford professorship is pretty helpful. The problems with academia Dwarkesh Patel 51:34 You mentioned in the book and elsewhere that there's a scarcity of people thinking about big picture questions—How contingent is history? How are people happy generally?—Are these questions that are too hard for other people? Or they don't care enough? What's going on? Why are there so few people talking about this? Will MacAskill 51:54 I just think there are many issues that are enormously important but are just not incentivized anywhere in the world. Companies don't incentivize work on them because they’re too big picture. Some of these questions are, “Is the future good, rather than bad? If there was a global civilizational collapse, would we recover? How likely is a long stagnation?” There’s almost no work done on any of these topics. Companies aren't interested too grand in scale. Academia has developed a culture where you don't tackle such problems. Partly, that's because they fall through the cracks of different disciplines. Partly because they seem too grand or too speculative. Academia is much more in the mode of making incremental gains in our understanding. It didn't always used to be that way. If you look back before the institutionalization of academic research, you weren't a real philosopher unless you had some grand unifying theory of ethics, political philosophy, metaphysics, logic, and epistemology. Probably the natural sciences too and economics. I'm not saying that all of academic inquiry should be like that. But should there be some people whose role is to really think about the big picture? Yes. Dwarkesh Patel 53:20 Will I be able to send my kids to MacAskill University? What's the status on that project? Will MacAskill 53:25 I'm pretty interested in the idea of creating a new university. There is a project that I've been in discussion about with another person who's fairly excited about making it happen. Will it go ahead? Time will tell. I think you can do both research and education far better than it currently exists. It's extremely hard to break in or creating something that's very prestigious because the leading universities are hundreds of years old. But maybe it's possible. I think it would could generate enormous amounts of value if we were able to pull it off. Dwarkesh Patel 54:10 Excellent, alright. So the book is What We Owe The Future. I understand pre-orders help a lot, right? It was such an interesting read. How often does somebody write a book about the questions they consider to be the most important even if they're not the most important questions? Big picture thinking, but also looking at very specific questions and issues that come up. Super interesting read. Will MacAskill 54:34 Great. Well, thank you so much! Dwarkesh Patel 54:38 Anywhere else they can find you? Or any other information they might need to know? Will MacAskill 54:39 Yeah, sure. What We Owe The Future is out on August 16 in the US and first of September in the United Kingdom. If you want to follow me on Twitter, I'm @WillMcCaskill. If you want to try and use your time or money to do good, Giving What We Can is an organization that encourages people to take a pledge to give a significant fraction of the income (10% or more) to the charities that do the most good. It has a list of recommended charities. 80,000 Hours—if you want to use your career to do good—is a place to go for advice on what careers have the biggest impact at all. They provide one-on-one coaching too. If you're feeling inspired and want to do good in the world, you care about future people and I want to help make their lives go better, then, as well as reading What We Owe The Future, Giving What We Can, and 80,000 hours are the sources you can go to and get involved. Dwarkesh Patel 55:33 Awesome, thanks so much for coming on the podcast! It was a lot of fun. Will MacAskill 54:39 Thanks so much, I loved it. Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe

Will MacAskill

Philosopher; Co-founder of Effective Altruism

University of Oxford

David Deutsch is the founder of the field of quantum computing and the author The Beginning of Infinity and The Fabric of Reality. Read me contra David on AI. Watch on YouTube. Listen on Apple Podcasts, Spotify, or any other podcast platform. Read the full transcript with helpful links here. Follow David on Twitter. Follow me on Twitter for updates on future podcasts. Timestamps (0:00:00) - Will AIs be smarter than humans?  (0:06:34) - Are intelligence differences immutable / heritable? (0:20:13) - IQ correletation of twins seperated at birth (0:27:12) - Do animals have bounded creativity? (0:33:32) - How powerful can narrow AIs be? (0:36:59) - Could you implant thoughts in VR? (0:38:49) - Can you simulate the whole universe? (0:41:23) - Are some interesting problems insoluble? (0:44:59) - Does America fail Popper's Criterion? (0:50:01) - Does finite matter mean there's no beginning of infinity? (0:53:16) - The Great Stagnation (0:55:34) - Changes in epistemic status is Popperianism (0:59:29) - Open ended science vs gain of function (1:02:54) - Contra Tyler Cowen on civilizational lifespan (1:07:20) - Fun criterion (1:14:16) - Does AGI through evolution require suffering? (1:18:01) - Would David enter the Experience Machine? (1:20:09) - (Against) Advice for young people Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe

David Deutsch

Physicist & Author

University of Oxford

Today we’re joined by Sandra Wacther, an associate professor and senior research fellow at the University of Oxford.  Sandra’s work lies at the intersection of law and AI, focused on what she likes to call “algorithmic accountability”. In our conversation, we explore algorithmic accountability in three segments, explainability/transparency, data protection, and bias, fairness and discrimination. We discuss how the thinking around black boxes changes when discussing applying regulation and law, as well as a breakdown of counterfactual explanations and how they’re created. We also explore why factors like the lack of oversight lead to poor self-regulation, and the conditional demographic disparity test that she helped develop to test bias in models, which was recently adopted by Amazon. The complete show notes for this episode can be found at twimlai.com/go/521.

Sandra Wachter

Associate Professor and Senior Research Fellow

University of Oxford

Topics covered in this episode: Powering the Python Package Index in 2021 The Leuven Star Atlas TI-84 Plus CE Python graphing calculator Python Package CI/CD with GitHub Actions SpaceX is using Python for prototyping their Starlink satellite software : A beginner’s guide to working with astronomical data Extras Joke See the full show notes for this episode on the website at pythonbytes.fm/234

Dr. Becky Smethurst

Astrophysicist

University of Oxford

See the full show notes for this episode on the website at talkpython.fm/303

Becky Smethurst

Astrophysicist & Science Communicator

University of Oxford

Dr. Simon Stringer. Obtained his Ph.D in mathematical state space control theory and has been a Senior Research Fellow at Oxford University for over 27 years. Simon is the director of the the Oxford Centre for Theoretical Neuroscience and Artificial Intelligence, which is based within the Oxford University Department of Experimental Psychology. His department covers vision, spatial processing, motor function, language and consciousness -- in particular -- how the primate visual system learns to make sense of complex natural scenes. Dr. Stringers laboratory houses a team of theoreticians, who are developing computer models of a range of different aspects of brain function. Simon's lab is investigating the neural and synaptic dynamics that underpin brain function. An important matter here is the The feature-binding problem which concerns how the visual system represents the hierarchical relationships between features. the visual system must represent hierarchical binding relations across the entire visual field at every spatial scale and level in the hierarchy of visual primitives. We discuss the emergence of self-organised behaviour, complex information processing, invariant sensory representations and hierarchical feature binding which emerges when you build biologically plausible neural networks with temporal spiking dynamics. 00:00:09 Tim Intro 00:09:31 Show kickoff 00:14:37 Hierarchical Feature binding and timing of action potentials 00:30:16 Hebb to Spike-timing-dependent plasticity (STDP) 00:35:27 Encoding of shape primitives 00:38:50 Is imagination working in the same place in the brain 00:41:12 Compare to supervised CNNs 00:45:59 Speech recognition, motor system, learning mazes 00:49:28 How practical are these spiking NNs 00:50:19 Why simulate the human brain 00:52:46 How much computational power do you gain from differential timings 00:55:08 Adversarial inputs 00:59:41 Generative / causal component needed? 01:01:46 Modalities of processing i.e. language 01:03:42 Understanding 01:04:37 Human hardware 01:06:19 Roadmap of NNs? 01:10:36 Intepretability methods for these new models 01:13:03 Won't GPT just scale and do this anyway? 01:15:51 What about trace learning and transformation learning 01:18:50 Categories of invariance 01:19:47 Biological plausibility https://www.youtube.com/watch?v=aisgNLypUKs

Simon Stringer

Senior Research Fellow & Director

University of Oxford

A dive into the sociological, operational, and tactical realities of this murky underworld, Lusthaus and de la Garza discuss who the players are, what they are motivated by, and specialize in—as well as how basic ideas like trust and anonymity function in a world where no one wants to get caught. How do criminal nicknames function as brand? Which countries tend to specialize in what kinds of crime, and why? And most of all, what changes when you begin to think of the business of cybercrime as an industry? Stay Updated: Find a16z on YouTube: YouTube Find a16z on X Find a16z on LinkedIn Listen to the a16z Show on Spotify Listen to the a16z Show on Apple Podcasts Follow our host: https://twitter.com/eriktorenberg Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures. Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

Jonathan Lusthaus

Researcher

University of Oxford

Even with the recent stock market rally, expectations are poor for a robust recovery in the U.S. So what does history teach us about what works and what doesn’t? Richard Werner is an economist at Linacre College at the University of Oxford, and the proponent of what he calls the “Quantity Theory of Credit.” On this episode, he tells us about what he learned studying years of the Japanese economy, and what it means for the current crisis. See omnystudio.com/listener for privacy information.

Richard Werner

Economist

University of Oxford

Richard Dawkins is an evolutionary biologist, and author of The Selfish Gene, The Blind Watchmaker, The God Delusion, The Magic of Reality, The Greatest Show on Earth, and his latest Outgrowing God. He is the originator and popularizer of a lot of fascinating ideas in evolutionary biology and science in general, including funny enough the introduction of the word meme in his 1976 book The Selfish Gene, which in the context of a gene-centered view of evolution is an exceptionally powerful idea. He is outspoken, bold, and often fearless in his defense of science and reason, and in this way, is one of the most influential thinkers of our time. Support this podcast by signing up with these sponsors: – Cash App – use code “LexPodcast” and download: – Cash App (App Store): https://apple.co/2sPrUHe – Cash App (Google Play): https://bit.ly/2MlvP5w EPISODE LINKS: Richard’s Website: https://www.richarddawkins.net/ Richard’s Twitter: https://twitter.com/RichardDawkins Richard’s Books: – Selfish Gene: https://amzn.to/34tpHQy – The Magic of Reality: https://amzn.to/3c0aqZQ – The Blind Watchmaker: https://amzn.to/2RqV5tH – The God Delusion: https://amzn.to/2JPrxlc – Outgrowing God: https://amzn.to/3ebFess – The Greatest Show on Earth: https://amzn.to/2Rp2j1h This conversation is part of the Artificial Intelligence podcast. If you would like to get more information about this podcast go to https://lexfridman.com/ai or connect with @lexfridman on Twitter, LinkedIn, Facebook, Medium, or YouTube where you can watch the video versions of these conversations. If you enjoy the podcast, please rate it 5 stars on Apple Podcasts, follow on Spotify, or support it on Patreon. Here’s the outline of the episode. On some podcast players you should be able to click the timestamp to jump to that time. OUTLINE: 00:00 – Introduction 02:31 – Intelligent life in the universe 05:03 – Engineering intelligence (are there shortcuts?) 07:06 – Is the evolutionary process efficient? 10:39 – Human brain and AGI 15:31 – Memes 26:37 – Does society need religion? 33:10 – Conspiracy theories 39:10 – Where do morals come from in humans? 46:10 – AI began with the ancient wish to forge the gods 49:18 – Simulation 56:58 – Books that influenced you 1:02:53 – Meaning of life

Richard Dawkins

Evolutionary Biologist and Author

University of Oxford

Roger Penrose is physicist, mathematician, and philosopher at University of Oxford. He has made fundamental contributions in many disciplines from the mathematical physics of general relativity and cosmology to the limitations of a computational view of consciousness. Support this podcast by signing up with these sponsors: – ExpressVPN at https://www.expressvpn.com/lexpod – Cash App – use code “LexPodcast” and download: – Cash App (App Store): https://apple.co/2sPrUHe – Cash App (Google Play): https://bit.ly/2MlvP5w EPISODE LINKS: Cycles of Time (book): https://amzn.to/39tXtpp The Emperor’s New Mind (book): https://amzn.to/2yfeVkD This conversation is part of the Artificial Intelligence podcast. If you would like to get more information about this podcast go to https://lexfridman.com/ai or connect with @lexfridman on Twitter, LinkedIn, Facebook, Medium, or YouTube where you can watch the video versions of these conversations. If you enjoy the podcast, please rate it 5 stars on Apple Podcasts, follow on Spotify, or support it on Patreon. Here’s the outline of the episode. On some podcast players you should be able to click the timestamp to jump to that time. OUTLINE: 00:00 – Introduction 03:51 – 2001: A Space Odyssey 09:43 – Consciousness and computation 23:45 – What does it mean to “understand” 31:37 – What’s missing in quantum mechanics? 40:09 – Whatever consciousness is, it’s not a computation 44:13 – Source of consciousness in the human brain 1:02:57 – Infinite cycles of big bangs 1:22:05 – Most beautiful idea in mathematics

Roger Penrose

Physicist, Mathematician, and Philosopher

University of Oxford