Building the Context Flywheel for AI Data Agents
Show notes
Summary In this episode Prukalpa Sankar, co-founder of Atlan, talks about what it takes to build a “context flywheel” for AI agents in data-intensive organizations. She explained why model intelligence alone isn’t enough to make AI useful in production, and how real performance depends on contextual intelligence: institutional knowledge, semantic meaning, procedural know-how, and access to the right tools. She also dug into how metadata catalogs are evolving into broader context layers that serve both humans and agents, and why agentic systems are changing the economics of metadata and governance work. Prakulpa shared Atlan’s perspective on bootstrapping context from existing systems such as warehouses, BI tools, query logs, and SaaS applications, then using simulation, traces, and human governance loops to improve agent accuracy over time. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management Your host is Tobias Macey and today I'm interviewing Prukalpa Sankar about strategies for building a context flywheel for your data agents Interview Introduction How did you get involved in the area of data management? You have spent several years working in the metadata catalog space with Atlan. What are the notable changes in scope, adoption, and application that you have seen since we last spoke (June 2022)? The recurring theme since the start of 2026 has been agentic augmentation of all engineering workflows, including data. How do you differentiate between data catalogs, semantic layers, agent memory, context layers, etc. when architecting an AI-powered data-oriented system? One of the perennial problems with data catalogs, business glossaries, master data management, etc. is the up-front investment required to get a real-world impact. How can agents help reduce the activation energy needed to get to that return on effort? One of the perennial problems in data engineering is fragmentation and siloing of data. This is exacerbated by AI systems due to the introduction of vector data as a new specialization. What are the forces that you are seeing play into the current set of tensions and the architectural primitives that we need to bring to bear to keep things maintainable? Since the introduction of transformer-based generative models we have been combating hallucinations. While we have made progress, it is still critical to ensure accuracy and trustworthiness when working with business data. What are the policy elements of governance and technical controls to ensure a high degree of confidence in agent-generated context and business semantics? What are the most interesting, innovative, or unexpected ways that you have seen teams build context layers for their agentic data workloads? What are the most interesting, unexpected, or challenging lessons that you have learned while working on business context engineering? When is agent-managed context the wrong choice? What are your predictions for the next set of architectural shifts that will be driven by the pressures of AI-powered systems? Contact Info LinkedIn Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? Closing Announcements Thank you for listening! Don't forget to check out our other shows. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. The AI Engineering Podcast is your guide to the fast-moving world of building AI systems. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you've learned something or tried out a project from the show then tell us about it! Email hosts@dataengineeringpodcast.com with your story. Links Atlan Atlan Context Lakehouse Iceberg Business Glossary Master Data Management Semantic Layer Cube.dev MCP == Model Context Protocol A2A == Agent to Agent Protocol Decision Traces Apache Doris StarRocks The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA
Guests
Analysis: done · source seed