How Replication Could Teach Machines What Good Science Looks Like — Edward Hughes
Can a machine learn the judgement that separates a plausible-looking result from a faithful experiment? Edward Hughes, Chief Scientist and co-founder of Inherent, joins Tim Scarfe to argue that creativity is not optimisation, and that the missing capability in AI is choosing which questions are worth asking.
SPONSOR:
---
Cyber Fund built the Monastery to help founders ship products that were impossible a year ago.
Apply now: https://cyber.fund
---
Edward makes the case that Move 37 was innovative rather than creative, and that the field, not the individual, decides what counts as a discovery. That reframing runs through Csikszentmihalyi, Deutsch and exaptation into open-endedness, where deceptive goals and imperfect world models turn out to be the point rather than the problem. The second half turns to the paper: Replica, a task space built by redacting figures from real papers, and Faraday, a 27-billion-parameter model trained to steer a frontier coding agent that then beats the frontier on held-out replications.
---
TIMESTAMPS:
00:00:00 Cold open: Move 37, Faraday and collective intelligence
00:01:08 Sponsor: CyberFund
00:01:46 Inherent's $50M raise and the road from string theory
00:09:14 Three timescales of learning: weights, context, culture
00:13:47 Move 37 was innovative, not creative: the field decides
00:20:39 Creativity as satisficing: the urinal and evolution
00:25:06 Exaptation and the Tristan chord: creativity in context
00:30:56 Coherence for whom? Deutsch's hard-to-vary explanations
00:35:53 Why copying is creative: Deutsch and the constraint engineer
00:42:27 Societies of agents and the strong Moravec paradox
00:45:51 Evaluate in hindsight: from Lean proofs to climate change
00:51:56 Picbreeder, local goals and why discovery needs deception
00:57:21 Spaghetti proofs, translation layers and superhuman Go
01:00:37 Does nature compress? Naturalness and real patterns
01:07:36 Why replicate? Replica's redacted figures and Faraday
01:12:31 Faraday beats Codex, Claude and GLM 5.2 on held-out tasks
01:15:31 Replication to innovation: how the Transformer happened
01:18:26 Deep replication: what Faraday learns from Voyager and GNoME
01:23:37 Can the AI scientist cheat? Goodharting the judge
01:29:09 Inside Replica: scale-down, 8xB300 runs, per-task rubrics
01:34:11 The RL crisis: getting GRPO to work with per-turn credit
01:39:43 Weights vs harnesses: AlphaEvolve, DGM and EvoTune
01:45:45 The recursive company: agents cross a phase transition
01:50:35 Collective intelligence and the electric dynamo
01:55:46 What replaces OKRs? Incumbents and the burden of knowledge
---
REFERENCES:
MLST Creativity Article:
https://archive.mlst.ai/read/why-creativity-cannot-be-interpolated
organization:
[00:01:47] Inherent
https://inherentlabs.ai/
other:
[00:20:51] Marcel Duchamp, Fountain
https://www.tate.org.uk/art/artworks/duchamp-fountain-t07573
[00:05:19] Human-Timescale Adaptation in an Open-Ended Task Space (Adaptive Agent)
https://arxiv.org/abs/2301.07608
[00:06:05] The AI Scientist
https://arxiv.org/abs/2408.06292
[00:12:13] Training AI Scientists to Replicate Research (Replica and Faraday)
https://arxiv.org/abs/2608.13331
[01:44:46] Evolutionary Principles in Self-Referential Learning
https://people.idsia.ch/~juergen/diploma.html
[01:59:33] Are Ideas Getting Harder to Find?
https://www.nber.org/papers/w23782
book:
[00:16:04] Creativity: Flow
https://search.worldcat.org/title/254487436
[00:26:22] Why Greatness Cannot Be Planned
https://link.springer.com/book/10.1007/978-3-319-15524-1
[00:33:03] The Beginning of Infinity
https://www.penguinrandomhouse.com/books/293575/the-beginning-of-infinity-by-david-deutsch/
[01:55:47] Laws of Knowledge
https://www.penguin.co.nz/books/the-infinite-alphabet-9780241655672
(Full list refs on YT/rescript)
---
RESCRIPT:
https://app.rescript.info/session/670296ba913761d0?share=6281911cac9bdbff637f10819d4d1e5c
AI:AM Highlights: Recursive Self-Improvement, Rushed and Vibe-Coded?
In this AI:AM Highlights episode, Nathan Labenz and Prakash revisit three live mornings with Louis Kirsch and Damon Falck of Inherent Laboratories, Vercel CTO Malte Ubl, Genesis Molecular AI CTO Sergey Edunov, Arm’s Mohamed Awad, David Li of Shenzhen Open Innovation Lab, and Q.ANT CEO Michael Förtsch. The central thread is that the meaningful unit of AI work is becoming a division of labor between models, from Inherent’s 27B “scientist” model using GPT-5.5 Codex for implementation to production pipelines that route taste, execution, security, and hardware coordination across different systems. The episode weighs why that shift matters for recursive self-improvement, verification of discoveries humans may not be able to check, model refusals in security work, and the infrastructure needed for agents that can act in the world. Its sharpest stake is whether frontier labs are scaling reinforcement learning and agentic workflows on top of reward environments and vendor pipelines that may be too rushed, noisy, or gameable to support the institutional self-improvement they are pursuing.
For full show notes, links, and references, read the episode page:
https://www.cognitiverevolution.ai/ai-am-highlights-recursive-self-improvement-rushed-and-vibe-coded/
Sponsors:
Mercury:
Mercury is the banking platform loved by 300,000+ entrepreneurs, with virtual cards and Spend controls for granular budgets, receipts, and low-risk AI agent purchases. Learn more and apply in minutes at https://mercury.com
Diffusion:
Diffusion helps organizations build custom AI software factories that scale business outcomes, not just outputs. Cognitive Revolution listeners get a 25% service credit on their first engagement at https://diffusion.io/tcr
Granola:
Granola is an AI-powered notepad that securely transcribes meetings and turns rough notes into clean, structured action items. Try it free at https://granola.ai/tcr
Deepgram Flux TTS:
Deepgram Flux TTS brings lifelike AI voices with real personalities that handle interruptions, pauses, and natural conversation. Try all the voices free through September 12 at https://deepgram.com/keep-talking
Claude:
Claude is the AI collaborator for problem solvers, helping with writing, coding, financial models, strategy, and more. Get started with Claude and explore Claude Pro at https://claude.ai/tcr
CHAPTERS:
(00:00) About the Episode
(01:16) Sponsor: Mercury
(02:58) Reward hacking environments (Part 1)
(15:12) Sponsors: Diffusion | Granola
(18:09) Reward hacking environments (Part 2)
(18:11) Inherent safety questions
(26:26) Recursive lab design (Part 1)
(31:24) Sponsors: Deepgram Flux TTS | Claude
(33:29) Recursive lab design (Part 2)
(44:34) Right model choices
(01:01:57) Hardware for agents
(01:19:44) Checking frontier agents
(01:33:46) China compute abundance
(01:44:21) Photonic compute stack
(01:53:17) Astra and AGI
(02:02:21) Slowdown and access
(02:06:53) Episode Outro
(02:09:53) Outro
PRODUCED BY:
https://aipodcast.ing
Claude Cooperates! Exploring Cultural Evolution in LLM Societies, with Aron Vallinder & Edward Hughes
In this episode, Edward Hughes, researcher at Google DeepMind, and Aron Vallinder, an independent researcher and PIBBSS fellow discuss their pioneering research on cultural evolution and cooperation among large language model agents. The conversation delves into the study's design, exploring how different AI models exhibit cooperative behavior in simulated environments, the implications of these findings for future AI development, and the potential societal impacts of autonomous AI agents. They elaborate on their experimental setup involving different LLMs like Claude, Gemini 1.5, and GPT-4.0 in a cooperative donor-recipient game, shedding light on how various AI models handle cooperation and their potential societal impacts. Key points include the importance of understanding externalities, the role of punishment and communication, and future research directions involving mixed-model societies and human-AI interactions. The episode invites listeners to engage in this fast-growing field, stressing the need for more hands-on research and empirical evidence to navigate the rapidly evolving AI landscape.
Link to Aron & Edward's research paper "Cultural Evolution of Cooperation among LLM
Agents"
SPONSORS:
Oracle Cloud Infrastructure (OCI): Oracle's next-generation cloud platform delivers blazing-fast AI and ML performance with 50% less for compute and 80% less for outbound networking compared to other cloud providers. OCI powers industry leaders like Vodafone and Thomson Reuters with secure infrastructure and application development capabilities. New U.S. customers can get their cloud bill cut in half by switching to OCI before March 31, 2024 at https://oracle.com/cognitive
NetSuite: Over 41,000 businesses trust NetSuite by Oracle, the #1 cloud ERP, to future-proof their operations. With a unified platform for accounting, financial management, inventory, and HR, NetSuite provides real-time insights and forecasting to help you make quick, informed decisions. Whether you're earning millions or hundreds of millions, NetSuite empowers you to tackle challenges and seize opportunities. Download the free CFO's guide to AI and machine learning at https://netsuite.com/cognitive
Shopify: Shopify is revolutionizing online selling with its market-leading checkout system and robust API ecosystem. Its exclusive library of cutting-edge AI apps empowers e-commerce businesses to thrive in a competitive market. Cognitive Revolution listeners can try Shopify for just $1 per month at https://shopify.com/cognitive
CHAPTERS:
(00:00) Teaser
(00:42) About the Episode
(03:26) Introduction
(03:40) The Rapid Evolution of AI
(04:58) Human Cooperation and Society
(07:03) Cultural Evolution and Stories
(08:39) Mechanisms of Cultural Evolution (Part 1)
(20:56) Sponsors: Oracle Cloud Infrastructure (OCI) | NetSuite
(23:35) Mechanisms of Cultural Evolution (Part 2)
(27:07) Experimental Setup: Donor Game (Part 1)
(37:35) Sponsors: Shopify
(38:55) Experimental Setup: Donor Game (Part 2)
(44:32) Exploring AI Societies: Claude, Gemini, and GPT-4
(45:50) Striking Graphical Differences
(48:08) Experiment Results and Implications
(50:54) Prompt Engineering and Cooperation
(57:40) Mixed Model Societies
(01:00:35) Future Research Directions
(01:03:10) Human-AI Interaction and Influence
(01:05:20) Complexifying AI Games
(01:18:14) Evaluations and Feedback Loops
(01:20:50) Open Source and AI Safety
(01:23:23) Reflections and Future Work
(01:30:04) Outro