Is Offense or Defense Dominant? FAR.AI's Adam Gleave on the AI Security Leaderboard
Show notes
FAR.AI co-founder and CEO Adam Gleave joins Nathan to discuss FAR.AI’s AI Security Leaderboard, the first systematic head-to-head evaluation of the misuse safeguards frontier developers actually ship. The findings expose a major measurement gap: while Claude Fable 5 and GPT-5.6 Sol withstood FAR.AI’s suite, Grok 4.5 and Gemini 3.1 Pro yielded hundreds of universal jailbreaks at low cost. Adam explains why many effective attacks look more like social engineering than advanced ML, why “jailbreak tax” should not be relied on for safety, and how FAR.AI scores whether a model is genuinely helping an attacker. The episode’s stakes are whether AI developers can measure and harden real deployed defenses before threat actors make routine use of increasingly capable systems. - FAR.AI AI Security Leaderboard: http://leaderboard.far.ai/ - People can e-mail owsa@far.ai if they're interested in the open-weight safety accelerator grantmaking program. For full show notes, links, and references, read the episode page: https://www.cognitiverevolution.ai/is-offense-or-defense-dominant-far-ai-s-adam-gleave-on-the-ai-security-leaderboard/ Sponsor: Claude: Claude by Anthropic is an AI collaborator that understands your workflow and helps you tackle research, writing, coding, and organization with deep context. Get started with Claude and explore Claude Pro at https://claude.ai/tcr CHAPTERS: (00:00) About the Episode (03:22) AI security leaderboard (07:56) Universal jailbreaks explained (16:26) Finding social jailbreaks (Part 1) (16:31) Sponsor: Claude (18:01) Finding social jailbreaks (Part 2) (30:48) Layered safeguard defenses (42:25) Uneven frontier safeguards (51:10) Sharing safety standards (01:00:30) Offense versus defense (01:08:50) Open-weight model safety (01:17:25) Control failure warnings (01:30:05) Coordination and risk (01:39:24) Episode Outro (01:42:52) Outro PRODUCED BY: https://aipodcast.ing SOCIAL LINKS: Website: https://www.cognitiverevolution.ai Twitter (Podcast): https://x.com/cogrev_podcast Twitter (Nathan): https://x.com/labenz LinkedIn: https://linkedin.com/in/nathanlabenz/ Youtube: https://youtube.com/@CognitiveRevolutionPodcast Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431 Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk
Guests
Analysis: done · source seed