Responsible AI in the Generative Era with Michael Kearns - #662
Today we’re joined by Michael Kearns, professor in the Department of Computer and Information Science at the University of Pennsylvania and an Amazon scholar. In our conversation with Michael, we discuss the new challenges to responsible AI brought about by the generative AI era. We explore Michael’s learnings and insights from the intersection of his real-world experience at AWS and his work in academia. We cover a diverse range of topics under this banner, including service card metrics, privacy, hallucinations, RLHF, and LLM evaluation benchmarks. We also touch on Clean Rooms ML, a secured environment that balances accessibility to private datasets through differential privacy techniques, offering a new approach for secure data handling in machine learning.
The complete show notes for this episode can be found at twimlai.com/go/662.
Service Cards and ML Governance with Michael Kearns - #610
Today we conclude our AWS re:Invent 2022 series joined by Michael Kearns, a professor in the department of computer and information science at UPenn, as well as an Amazon Scholar. In our conversation, we briefly explore Michael’s broader research interests in responsible AI and ML governance and his role at Amazon. We then discuss the announcement of service cards, and their take on “model cards” at a holistic, system level as opposed to an individual model level. We walk through the information represented on the cards, as well as explore the decision-making process around specific information being omitted from the cards. We also get Michael’s take on the years-old debate of algorithmic bias vs dataset bias, what some of the current issues are around this topic, and what research he has seen (and hopes to see) addressing issues of “fairness” in large language models.
The complete show notes for this episode can be found at twimlai.com/go/610.
Michael Kearns on Privacy
Michael Kearns, a computer scientist professor at the University of Pennsylvania and an Amazon scholar talks about differential privacy, how Amazon's research approach differs from its peers, and how AI will eventually permeate all aspects of our lives.
Michael Kearns: Algorithmic Fairness, Bias, Privacy, and Ethics in Machine Learning
Michael Kearns is a professor at University of Pennsylvania and a co-author of the new book Ethical Algorithm that is the focus of much of our conversation, including algorithmic fairness, bias, privacy, and ethics in general. But, that is just one of many fields that Michael is a world-class researcher in, some of which we touch on quickly including learning theory or theoretical foundations of machine learning, game theory, algorithmic trading, quantitative finance, computational social science, and more.
This conversation is part of the Artificial Intelligence podcast. If you would like to get more information about this podcast go to https://lexfridman.com/ai or connect with @lexfridman on Twitter, LinkedIn, Facebook, Medium, or YouTube where you can watch the video versions of these conversations. If you enjoy the podcast, please rate it 5 stars on Apple Podcasts or support it on Patreon. This episode is sponsored by Pessimists Archive podcast. Here’s the outline with timestamps for this episode (on some players you can click on the timestamp to jump to that point in the episode):
00:00 – Introduction
02:45 – Influence from literature and journalism
07:39 – Are most people good?
13:05 – Ethical algorithm
24:28 – Algorithmic fairness of groups vs individuals
33:36 – Fairness tradeoffs
46:29 – Facebook, social networks, and algorithmic ethics
58:04 – Machine learning
58:05 – Machine learning
59:19 – Algorithm that determines what is fair
1:01:25 – Computer scientists should think about ethics
1:05:59 – Algorithmic privacy
1:11:50 – Differential privacy
1:19:10 – Privacy by misinformation
1:22:31 – Privacy of data in society
1:27:49 – Game theory
1:29:40 – Nash equilibrium
1:30:35 – Machine learning and game theory
1:34:52 – Mutual assured destruction
1:36:56 – Algorithmic trading
1:44:09 – Pivotal moment in graduate school
