When AI Decides You're a Threat — Brad Carson
Show notes
Brad Carson was the Army's General Counsel, served two terms in Congress and was Acting Under Secretary of Defense for Personnel and Readiness. He now heads Americans for Responsible Innovation, the AI-policy advocacy group he co-founded. Keith Duggar spends roughly eighty minutes pushing back. SPONSOR: --- Cyber Fund built the Monastery to help founders ship products that were impossible a year ago. Applications for Batch 1 are now open. Apply now: https://cyber.fund --- Carson's whole case rests on one line: the genie is not out of the bottle. We have pulled dangerous tech back before. Asilomar halted recombinant DNA in 1975, and the West still controls the chips AI runs on. Calling it unstoppable, he says, is the most dangerous idea in the room. Then Keith drags him somewhere darker. A Palantir heat map scores you 0.73 on whether you are a combatant, and a strike follows. The model is wrong some accepted share of the time, and when it is, nobody answers for it. You cannot court-martial a model, and not even the interpretability researchers can say why it picked you. — Note: after recording, we learned that Americans for Responsible Innovation is backed by EA-aligned philanthropy (not sponsored) --- TIMESTAMPS: 00:00:00 From the Pentagon to AI governance 00:04:52 Regulatory capture vs Silicon Valley networks 00:07:56 Transparency and the Claude tier changes 00:09:40 Tort liability when AI tools cause harm 00:13:40 AI is a product, not a person 00:16:01 Children, suicide, and the suicide business 00:19:59 Opaque neural nets and the law of war 00:25:54 Probabilistic targeting and the death of accountability 00:28:47 The arms race fallacy: Asilomar and restraint 00:34:02 Talking to China: track 2 talks and chip leverage 00:39:45 Air power never wins: capital for labour 00:43:29 Anthropic vs the Department of War 00:51:29 Concentration, open source, and brain drain 01:00:18 DeepSeek, Chinese culture, and AI as diplomacy 01:12:25 Upskilling Congress and why public trust matters --- REFERENCES: organization: [00:02:45] ICRC position on autonomous weapons https://www.icrc.org/en/law-and-policy/autonomous-weapons [00:05:22] Americans for Responsible Innovation (ARI) https://ari.us [00:07:20] Andreessen Horowitz (a16z) https://a16z.com/ [01:16:05] Office of Technology Assessment https://en.wikipedia.org/wiki/Office_of_Technology_Assessment other: [00:03:35] Beneficial AGI 2019 Conference (Future of Life Institute, Puerto Rico) https://futureoflife.org/event/beneficial-agi-2019/ [00:18:30] Section 230 of the Communications Decency Act https://en.wikipedia.org/wiki/Section_230 [00:19:59] Lethal Autonomous Weapons (LAWS) https://en.wikipedia.org/wiki/Lethal_autonomous_weapon [00:31:35] Strategic Arms Limitation Talks (SALT) https://en.wikipedia.org/wiki/Strategic_Arms_Limitation_Talks [00:32:28] Asilomar Conference on Recombinant DNA (1975) https://en.wikipedia.org/wiki/Asilomar_Conference_on_Recombinant_DNA [00:39:45] The New Iron Triangle (ARI policy byte) https://ari.us/policy-bytes/the-new-iron-triangle/ [00:48:05] Defense Production Act https://en.wikipedia.org/wiki/Defense_Production_Act person: [00:03:35] Anthony Aguirre https://en.wikipedia.org/wiki/Anthony_Aguirre [00:06:48] Dean Ball — Hyperdimensional https://www.hyperdimensional.co/ [00:23:13] Neel Nanda — mechanistic interpretability https://www.neelnanda.io/ [00:36:02] Jack Clark (Anthropic) on Conversations with Tyler https://conversationswithtyler.com/episodes/jack-clark/ [00:39:15] Robert Trager — Centre for the Governance of AI https://www.governance.ai/team/robert-trager [00:41:55] Giulio Douhet https://en.wikipedia.org/wiki/Giulio_Douhet [01:15:05] Don Beyer (US Congress) https://en.wikipedia.org/wiki/Don_Beyer tool: [00:22:19] Phalanx CIWS https://en.wikipedia.org/wiki/Phalanx_CIWS --- ReScript: https://app.rescript.info/public/share/9405ff35c0215b7cdae6402d41284171 https://app.rescript.info/api/public/sessions/0a6c081b8e5fe413/pdf
Guests
Analysis: done · source seed