American Express

Merck

Chairman & Managing Director

Data Engineering Podcast/Oct 10, 2022/41m

Investing In Understanding The Customer Journey At American Express

Summary For any business that wants to stay in operation, the most important thing they can do is understand their customers. American Express has invested substantial time and effort in their Customer 360 product to achieve that understanding. In this episode Purvi Shah, the VP of Enterprise Big Data Platforms at American Express, explains how they have invested in the cloud to power this visibility and the complex suite of integrations they have built and maintained across legacy and modern systems to make it possible. Announcements Hello and welcome to the Data Engineering Podcast, the show about modern data management When you’re ready to build your next pipeline, or want to test out the projects you hear about on the show, you’ll need somewhere to deploy it, so check out our friends at Linode. With their new managed database service you can launch a production ready MySQL, Postgres, or MongoDB cluster in minutes, with automated backups, 40 Gbps connections from your application hosts, and high throughput SSDs. Go to dataengineeringpodcast.com/linode today and get a $100 credit to launch a database, create a Kubernetes cluster, or take advantage of all of their other services. And don’t forget to thank them for their continued support of this show! You wake up to a Slack message from your CEO, who’s upset because the company’s revenue dashboard is broken. You’re told to fix it before this morning’s board meeting, which is just minutes away. Enter Metaplane, the industry’s only self-serve data observability tool. In just a few clicks, you identify the issue’s root cause, conduct an impact analysis⁠—and save the day. Data leaders at Imperfect Foods, Drift, and Vendr love Metaplane because it helps them catch, investigate, and fix data quality issues before their stakeholders ever notice they exist. Setup takes 30 minutes. You can literally get up and running with Metaplane by the end of this podcast. Sign up for a free-forever plan at dataengineeringpodcast.com/metaplane, or try out their most advanced features with a 14-day free trial. Mention the podcast to get a free "In Data We Trust World Tour" t-shirt. RudderStack helps you build a customer data platform on your warehouse or data lake. Instead of trapping data in a black box, they enable you to easily collect customer data from the entire stack and build an identity graph on your warehouse, giving you full visibility and control. Their SDKs make event streaming from any app or website easy, and their state-of-the-art reverse ETL pipelines enable you to send enriched data to any cloud tool. Sign up free… or just get the free t-shirt for being a listener of the Data Engineering Podcast at dataengineeringpodcast.com/rudder. Data teams are increasingly under pressure to deliver. According to a recent survey by Ascend.io, 95% in fact reported being at or over capacity. With 72% of data experts reporting demands on their team going up faster than they can hire, it’s no surprise they are increasingly turning to automation. In fact, while only 3.5% report having current investments in automation, 85% of data teams plan on investing in automation in the next 12 months. 85%!!! That’s where our friends at Ascend.io come in. The Ascend Data Automation Cloud provides a unified platform for data ingestion, transformation, orchestration, and observability. Ascend users love its declarative pipelines, powerful SDK, elegant UI, and extensible plug-in architecture, as well as its support for Python, SQL, Scala, and Java. Ascend automates workloads on Snowflake, Databricks, BigQuery, and open source Spark, and can be deployed in AWS, Azure, or GCP. Go to dataengineeringpodcast.com/ascend and sign up for a free trial. If you’re a data engineering podcast listener, you get credits worth $5,000 when you become a customer. Your host is Tobias Macey and today I’m interviewing Purvi Shah about building the Customer 360 data product for American Express and migrating their enterprise data platform to the cloud Interview Introduction How did you get involved in the area of data management? Can you describe what the Customer 360 project is and the story behind it? What are the types of questions and insights that the C360 project is designed to answer? Can you describe the types of information and data sources that you are relying on to feed this project? What are the different axes of scale that you have had to address in the design and architecture of the C360 project? (e.g. geographical, volume/variety/velocity of data, scale of end-user access and data manipulation, etc.) What are some of the challenges that you have had to address in order to build and maintain the map between organizational and technical requirements/semantics in the platform? What were some of the early wins that you targeted, and how did the lessons from those successes drive the product design going forward? Can you describe the platform architecture for your data systems that are powering the C360 product? How have the design/goals/requirements of the system changed since you first started working on it? How have you approached the integration and migration of legacy data systems and assets into this new platform? What are some of the ongoing maintenance challenges that the legacy platforms introduce? Can you describe how you have approached the question of data quality/observability and the validation/verification of the generated assets? What are the aspects of governance and access control that you need to deal with being part of a financial institution? Now that the C360 product has been in use for a few years, what are the strategic and tactical aspects of the ongoing evolution and maintenance of the product which you have had to address? What are the most interesting, innovative, or unexpected ways that you have seen the C360 product used? What are the most interesting, unexpected, or challenging lessons that you have learned while working on C360 for American Express? When is a C360 project the wrong choice? What do you have planned for the future of C360 and enterprise data platforms at American Express? Contact Info LinkedIn Parting Question From your perspective, what is the biggest gap in the tooling or technology for data management today? Closing Announcements Thank you for listening! Don’t forget to check out our other shows. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. The Machine Learning Podcast helps you go from idea to production with machine learning. Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes. If you’ve learned something or tried out a project from the show then tell us about it! Email hosts@dataengineeringpodcast.com) with your story. To help other people find the show please leave a review on Apple Podcasts and tell your friends and co-workers Links Data Stewards Hadoop SBA Paycheck Protection The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA Support Data Engineering Podcast

Purvi Shah

VP of Enterprise Big Data Platforms

Masters of Scale/Jun 2, 2022/29m

Rapid Response: Why business must be a force for good, w/former American Express CEO Ken Chenault

"Corporations can be a force for good – and they can also be very successful," says Ken Chenault, chair of investment firm General Catalyst and former longtime CEO of American Express. During the past year, Ken has been an outspoken advocate for business leaders to actively engage in societal matters. After George Floyd’s death, he and Merck CEO Ken Frazier launched OneTen, a coalition to create 1 million jobs for Black Americans that now includes major brands from IBM to Nike to Walmart. Chenault has pushed corporate chiefs to use their leverage to protect voting rights, and he's come out with a platform for Responsible Innovation that puts what he calls “social due diligence” alongside financial due diligence. Standing up for what's right, Ken says, fuels long-term success: "The most important thing is the quest for truth, character, and values." Read a transcript of this interview at: mastersofscale.com Subscribe to the Masters of Scale weekly newsletter: https://mastersofscale.com/subscribe See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

Chairman & Managing Director

TED Talks Daily/May 19, 2022/13m

How great leaders innovate -- responsibly | Ken Chenault

In times of uncertainty, leaders have a responsibility to inspire hope. Sharing hard-won wisdom, business leader Ken Chenault talks about what it takes to enact positive, enduring change -- and why it's more important than ever to invest in responsible innovation that uplifts people and centers equality and fairness. (This conversation was hosted by TED current affairs curator Whitney Pennington Rodgers.) Hosted on Acast. See acast.com/privacy for more information.

Chairman & Managing Director

Masters of Scale/Jun 17, 2021/27m

Rapid Response: Why business must be a force for good, w/former American Express CEO Ken Chenault

Chairman & Managing Director

Practical AI/Jan 11, 2021/49m

The $1 trillion dollar ML model 💵

American Express is running what is perhaps the largest commercial ML model in the world; a model that automates over 8 billion decisions, ingests data from over $1T in transactions, and generates decisions in mere milliseconds or less globally. Madhurima Khandelwal, head of AMEX AI Labs, joins us for a fascinating discussion about scaling research and building robust and ethical AI-driven financial applications. Sponsors: Code-ish by Heroku – A podcast from the team at Heroku, exploring code, technology, tools, tips, and the life of the developer. Check out episode 98 and episode 99 for insights on the ethical and technical sides of deep fakes. Subscribe on Apple Podcasts and Spotify. Changelog++ – You love our content and you want to take it to the next level by showing your support. We’ll take you closer to the metal with no ads, extended episodes, outtakes, bonus content, a deep discount in our merch store (soon), and more to come. Let’s do this! Fastly – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com. LaunchDarkly – Test in production! Deploy code at any time, even if a feature isn’t ready to be released to your users. Wrap code in feature flags to get the safety to test new features and infrastructure in prod without impacting the wrong end users. Featuring: Madhurima Khandelwal – LinkedIn Chris Benson – Website, GitHub, LinkedIn, X Daniel Whitenack – Website, GitHub, X Show Notes: AMEX AI Labs research O’Reilly article about “doing good data science” Upcoming Events: Register for upcoming webinars here!

Madhurima Khandelwal

Head of AI Labs