Reassessing the LLM Landscape & Summoning Ghosts
<p>What are the current techniques being employed to improve the performance of LLM-based systems? How is the industry shifting from post-training towards context engineering and multi-agent orchestration? This week on the show, Jodie Burchell, data scientist and Python Advocacy Team Lead at JetBrains, returns to discuss the current AI coding landscape.</p>
<p>In our last conversation, Jodie covered how LLMs were approaching the limits of scaling laws. This time, we recap last year’s big focus on reasoning models and a post-training method called “reinforcement learning from verifiable rewards” (RLVR). We also cover test-time compute, where models spend more time reasoning through steps and considering multiple approaches to solve a problem.</p>
<p>We touch on Agent Context Protocol (ACP), agent orchestration layers, and context engineering. We also share some concerns about the hype cycle, maintaining all that code being generated, and running local models.</p>
<div class="alert alert-primary" role="alert">
<p><strong>Course Spotlight:</strong> <a href="https://realpython.com/courses/vector-databases-embeddings-chromadb/">Vector Databases and Embeddings With ChromaDB</a></p>
<p>Learn how to use ChromaDB, an open-source vector database, to store embeddings and give context to large language models in Python.</p>
</div>
<p>Topics:</p>
<ul>
<li>00:00:00 – Introduction</li>
<li>00:02:02 – Build a Language-Learning Agent course</li>
<li>00:02:55 – Update on the past six months of LLMs</li>
<li>00:05:32 – Reinforcement Learning From Verifiable Rewards</li>
<li>00:07:32 – Test Time Compute</li>
<li>00:08:36 – 2025 and the rise of agents</li>
<li>00:14:24 – Benchmarks shifting</li>
<li>00:15:23 – Andrew Karpathy and jagged intelligence</li>
<li>00:19:16 – Not evolving or growing animals but summoning ghosts</li>
<li>00:23:34 – Diminishing gains in newer models</li>
<li>00:24:23 – Context Engineering</li>
<li>00:35:01 – Multi-agent systems and diversity of models</li>
<li>00:36:56 – Video Course Spotlight</li>
<li>00:38:34 – Current generation of coding agents</li>
<li>00:44:00 – Fast vs deep reasoning</li>
<li>00:45:18 – Agent Context Protocol</li>
<li>00:50:19 – Working through the hype cycle</li>
<li>00:55:43 – Open-source contribution pollution</li>
<li>00:57:21 – Local models</li>
<li>00:58:36 – Rick Beato comparing how the music industry failed</li>
<li>01:08:41 – LLMs are an amazing development</li>
<li>01:11:33 – Keynote talk on AI summers and winters</li>
<li>01:12:45 – PyCon US and EuroPython</li>
<li>01:14:11 – Thanks and goodbye</li>
</ul>
<p>Show Links:</p>
<ul>
<li><a href="https://www.youtube.com/watch?v=j4sNAwrx3kc&t=86s">AI Agent Course - Build a Language‑Learning Agent with OpenAI, LangGraph, Ollama & MCP - YouTube</a></li>
<li><a href="https://realpython.com/podcasts/rpp/264/">Episode #264: Large Language Models on the Edge of the Scaling Laws</a></li>
<li><a href="https://arxiv.org/abs/2506.14245">Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs</a></li>
<li><a href="https://labelbox.com/solutions/reinforcement-learning-with-verifiable-rewards/">Reinforcement learning with verifiable rewards (RLVR)</a></li>
<li><a href="https://huggingface.co/blog/Kseniase/testtimecompute">What is test-time compute and how to scale it?</a></li>
<li><a href="https://en.wikipedia.org/wiki/Overfitting">Overfitting - Wikipedia</a></li>
<li><a href="https://karpathy.bearblog.dev/year-in-review-2025/">2025 LLM Year in Review - karpathy</a></li>
<li><a href="https://karpathy.bearblog.dev/animals-vs-ghosts/">Animals vs Ghosts - karpathy</a></li>
<li><a href="https://arxiv.org/abs/2505.14569">Agent Context Protocols Enhance Collective Inference</a></li>
<li><a href="https://wagtail.org/blog/open-source-ai-we-use-to-work-on-wagtail/">Open source AI we use to work on Wagtail - Wagtail CMS</a></li>
<li><a href="https://www.youtube.com/watch?v=iRqpsCHqLUI">LLMs for Devs: Model Selection, Hallucinations, Agents, AGI – Jodie Burchell - The Marco Show</a></li>
<li><a href="https://t-redactyl.io/talks/2025-06-29-can-you-trust-your-large-language-model/">Keynote - Can you trust your (large language) model? - Standard error</a></li>
<li><a href="https://pydantic.dev/articles/the-human-in-the-loop-is-tired">The Human-in-the-Loop is Tired</a></li>
<li><a href="https://www.youtube.com/watch?v=YTLnnoZPALI">How AI Will Fail Like The Music Industry - YouTube</a></li>
<li><a href="https://www.theringer.com/podcasts/plain-english-with-derek-thompson/2026/03/17/yes-ai-is-a-bubble-there-is-no-question">“Yes, AI Is a Bubble. There Is No Question.” - The Ringer</a></li>
<li><a href="https://www.youtube.com/watch?v=x5s_gsu9Hgs">Keynote: AI is having its moment … again - Jodie Burchell - NDC Copenhagen 2025</a></li>
<li><a href="https://us.pycon.org/2026/">PyCon US 2026</a></li>
<li><a href="https://ep2026.europython.eu/">EuroPython 2026 - July 13th-19th 2026 - Kraków, Poland</a></li>
<li><a href="https://bsky.app/profile/t-redactyl.bsky.social">Jodie Burchell (@t-redactyl.bsky.social) — Bluesky</a></li>
<li><a href="https://t-redactyl.io/">Standard error</a></li>
</ul>
<p>Level up your Python skills with our expert-led courses:</p>
<ul>
<li><a href="https://realpython.com/courses/vector-databases-embeddings-chromadb/">Vector Databases and Embeddings With ChromaDB</a></li>
<li><a href="https://realpython.com/courses/getting-started-claude-code/">Getting Started With Claude Code</a></li>
<li><a href="https://realpython.com/courses/getting-started-google-gemini-cli/">Getting Started With Google Gemini CLI</a></li>
</ul> <p><a rel="payment" href="https://realpython.com/join">Support the podcast & join our community of Pythonistas</a></p>
#532: 2025 Python Year in Review
Python in 2025 is in a delightfully refreshing place: the GIL's days are numbered, packaging is getting sharper tools, and the type checkers are multiplying like gremlins snacking after midnight. On this episode, we have an amazing panel to give us a range of perspectives on what matter in 2025 in Python. We have Barry Warsaw, Brett Cannon, Gregory Kapfhammer, Jodie Burchell, Reuven Lerner, and Thomas Wouters on to give us their thoughts.
Episode sponsors
Seer: AI Debugging, Code talkpython26
Talk Python Courses
Links from the show
Python Software Foundation (PSF): www.python.org
PEP 810: Explicit lazy imports: peps.python.org
PEP 779: Free-threaded Python is officially supported: peps.python.org
PEP 723: Inline script metadata: peps.python.org
PyCharm: www.jetbrains.com
JetBrains: www.jetbrains.com
Visual Studio Code: code.visualstudio.com
pandas: pandas.pydata.org
PydanticAI: ai.pydantic.dev
OpenAI API docs: platform.openai.com
uv: docs.astral.sh
Hatch: github.com
PDM: pdm-project.org
Poetry: python-poetry.org
Project Jupyter: jupyter.org
JupyterLite: jupyterlite.readthedocs.io
PEP 690: Lazy Imports: peps.python.org
PyTorch: pytorch.org
Python concurrent.futures: docs.python.org
Python Package Index (PyPI): pypi.org
EuroPython: tickets.europython.eu
TensorFlow: www.tensorflow.org
Keras: keras.io
PyCon US: us.pycon.org
NumFOCUS: numfocus.org
Python discussion forum (discuss.python.org): discuss.python.org
Language Server Protocol: microsoft.github.io
mypy: mypy-lang.org
Pyright: github.com
Pylance: marketplace.visualstudio.com
Pyrefly: github.com
ty: github.com
Zuban: docs.zubanls.com
Jedi: jedi.readthedocs.io
GitHub: github.com
PyOhio: www.pyohio.org
Watch this episode on YouTube: youtube.com
Episode #532 deep-dive: talkpython.fm/532
Episode transcripts: talkpython.fm
Theme Song: Developer Rap
🥁 Served in a Flask 🎸: talkpython.fm/flasksong
---== Don't be a stranger ==---
YouTube: youtube.com/@talkpython
Bluesky: @talkpython.fm
Mastodon: @talkpython@fosstodon.org
X.com: @talkpython
Michael on Bluesky: @mkennedy.codes
Michael on Mastodon: @mkennedy@fosstodon.org
Michael on X.com: @mkennedy
Large Language Models on the Edge of the Scaling Laws
<p>What’s happening with the latest releases of large language models? Is the industry hitting the edge of the scaling laws, and do the current benchmarks provide reliable performance assessments? This week on the show, Jodie Burchell returns to discuss the current state of LLM releases.</p>
<p>The most recent release of GPT-5 has been a wake-up call for the LLM industry. We discuss how the current scaling of these systems is reaching a diminishing edge. Jodie also shares how many AI model assessments and benchmarks are flawed. We also take a sober look at the productivity gains from using these tools for software development within companies.</p>
<p>We discuss how newer developers should consider additional factors when looking at the current job market. Jodie digs into how economic changes and rising interest rates are influencing layoffs and hiring freezes. Then we share a wide collection of resources for you to continue exploring these topics.</p>
<p>This episode is sponsored by InfluxData.</p>
<div class="alert alert-primary" role="alert">
<p><strong>Course Spotlight:</strong> <a href="https://realpython.com/courses/closures-examples-use-cases/">Exploring Python Closures: Examples and Use Cases</a></p>
<p>Learn about Python closures: function-like objects with extended scope used for decorators, factories, and stateful functions.</p>
</div>
<p>Topics:</p>
<ul>
<li>00:00:00 – Introduction</li>
<li>00:03:00 – Recent conferences and talks</li>
<li>00:04:18 – What’s going on with LLMs?</li>
<li>00:06:06 – What happened with the GPT-5 release?</li>
<li>00:08:14 – Simon Willison - 2025 in LLMs so far</li>
<li>00:09:00 – How did we get here?</li>
<li>00:10:37 – OpenAI’s and scaling laws</li>
<li>00:12:25 – Pivoting to post-training</li>
<li>00:16:01 – Some history of AI eras</li>
<li>00:17:54 – Issues with measuring performance and benchmarks</li>
<li>00:22:19 – Chatbot Arena</li>
<li>00:24:06 – Languages are finite</li>
<li>00:26:22 – LLMs and the illusion of humanity</li>
<li>00:30:41 – Sponsor: InfluxData</li>
<li>00:31:34 – Types of solutions to move past these limits</li>
<li>00:36:57 – Does AI actually boost developer productivity?</li>
<li>00:44:19 – Agentic Al Programming with Python</li>
<li>00:48:02 – Results of non-programmers vibe coding</li>
<li>00:50:18 – Back to the concept of overfitting</li>
<li>00:52:52 – The money involved in training</li>
<li>00:56:50 – Video Course Spotlight</li>
<li>00:58:21 – Deepseek and new methods of training</li>
<li>01:01:02 – Quantizing and fitting on a local machine</li>
<li>01:04:48 – The layoffs and the economic changes</li>
<li>01:10:32 – AI implementation failures</li>
<li>01:21:01 – Don’t doubt yourself as a developer</li>
<li>01:24:06 – What are you excited about in the world of Python?</li>
<li>01:25:39 – What do you want to learn next?</li>
<li>01:26:42 – What’s the best way to follow your work online?</li>
<li>01:27:04 – Thanks and goodbye</li>
</ul>
<p>Survey:</p>
<ul>
<li><a href="https://tally.so/r/nWbKgJ">Listener Survey - Help Shape the Future of the Real Python Podcast</a></li>
</ul>
<p>Show Links:</p>
<ul>
<li><a href="https://ep2025.europython.eu/">EuroPython 2025 - July 14th-20th 2025 - Prague, Czech Republic & Remote</a></li>
<li><a href="https://realpython.com/podcasts/rpp/232/">Episode #232: Exploring Modern Sentiment Analysis Approaches in Python</a></li>
<li><a href="https://garymarcus.substack.com/p/gpt-5-overdue-overhyped-and-underwhelming">GPT-5: Overdue, overhyped and underwhelming. And that’s not the worst of it.</a></li>
<li><a href="https://spectrum.ieee.org/gpt-5-trough-of-disillusionment">GPT 5’s Rocky Launch Highlights AI Disillusionment - IEEE Spectrum</a></li>
<li><a href="https://www.youtube.com/watch?v=YpY83-kA7Bo">2025 in LLMs so far, illustrated by Pelicans on Bicycles — Simon Willison</a></li>
<li><a href="https://research.google/pubs/attention-is-all-you-need/">Attention is All You Need - Google</a></li>
<li><a href="https://openai.com/index/scaling-laws-for-neural-language-models/">Scaling laws for neural language models - OpenAI</a></li>
<li><a href="https://calnewport.com/what-if-ai-doesnt-get-much-better-than-this/">What if AI Doesn’t Get Much Better Than This? - Cal Newport</a></li>
<li><a href="https://www.latimes.com/business/story/2025-08-20/say-farewell-to-the-ai-bubble-and-get-ready-for-the-crash">Hiltzik: AI hype is fading fast - Los Angeles Times</a></li>
<li><a href="https://www.youtube.com/watch?v=tbDDYKRFjhk">Does AI Actually Boost Developer Productivity? (100k Devs Study) - Yegor Denisov-Blanch, Stanford - YouTube</a></li>
<li><a href="https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/">Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity - METR</a></li>
<li><a href="https://www.businessinsider.com/amazon-cloud-chief-replacing-junior-staff-ai-matt-garman-2025-8">Amazon Cloud Chief: Replacing Junior Staff With AI Is ‘Dumbest’ Idea - Business Insider</a></li>
<li><a href="https://www.evidentlyai.com/llm-guide/llm-benchmarks#reasoning-and-language-understanding-benchmarks">20 LLM evaluation benchmarks and how they work</a></li>
<li><a href="https://arxiv.org/abs/2009.03300">MMLU - Measuring Massive Multitask Language Understanding</a></li>
<li><a href="https://arxiv.org/abs/1905.07830">HellaSwag: Can a Machine Really Finish Your Sentence?</a></li>
<li><a href="https://en.m.wikipedia.org/wiki/Mechanical_Turk">Mechanical Turk - Wikipedia</a></li>
<li><a href="https://www.mturk.com/">Amazon Mechanical Turk</a></li>
<li><a href="https://lmarena.ai/?leaderboard">Chatbot Arena - LMArena</a></li>
<li><a href="https://customgpt.ai/llm-reasoning-vs-memorization/">LLMs Can’t Reason - The Reversal Curse, The Alice In Wonderland Test, And The ARC - AGI Challenge - CustomGPT</a></li>
<li><a href="https://www.youtube.com/watch?v=kqJ7rZHFx84">Mirror, mirror: LLMs and the illusion of humanity - Jodie Burchell - NDC Oslo 2024 - YouTube</a></li>
<li><a href="https://arxiv.org/pdf/2508.01191">Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens</a></li>
<li><a href="https://www.youtube.com/watch?v=TUjQuC4ugak&list=WL&index=1">Context Rot: How Increasing Input Tokens Impacts LLM Performance - YouTube</a></li>
<li><a href="https://www.youtube.com/watch?v=tbDDYKRFjhk">Does AI Actually Boost Developer Productivity? (100k Devs Study) - Yegor Denisov-Blanch, Stanford - YouTube</a></li>
<li><a href="https://www.teamblind.com/post/aws-ceo-says-no-more-programmers-in-2-years-wqarugct">AWS CEO says no more programmers in 2 years - Tech Industry - Blind</a></li>
<li><a href="https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/">MIT report: 95% of generative AI pilots at companies are failing - Fortune</a></li>
<li><a href="https://talkpython.fm/episodes/show/517/agentic-al-programming-with-python">Agentic Al Programming with Python - Talk Python To Me Podcast</a></li>
<li><a href="https://www.theverge.com/the-vergecast/759768/gpt5-backlash-vibe-coding-attempts-corporate-stunts">Vibe coding through the GPT-5 mess - The Verge</a></li>
<li><a href="https://en.wikipedia.org/wiki/Overfitting">Overfitting - Wikipedia</a></li>
<li><a href="https://www.youtube.com/watch?v=zjkBMFhNj_g&t=107s">Andrej Karpathy - Busy Person’s Intro to LLMs - YouTube</a></li>
<li><a href="https://andrewstiefel.com/ai-layoff-myth/">AI Isn’t Taking Your Job – The Economy Is - Andrew Stiefel</a></li>
<li><a href="https://www.abc.net.au/news/2025-08-21/cba-backtracks-on-ai-job-cuts-as-chatbot-lifts-call-volumes/105679492">Commonwealth Bank backtracks on AI job cuts, apologizes for ‘error’ as call volumes rise - ABC News</a></li>
<li><a href="https://www.entrepreneur.com/business-news/klarna-ceo-reverses-course-by-hiring-more-humans-not-ai/491396">Klarna CEO Reverses Course By Hiring More Humans, Not AI | Entrepreneur</a></li>
<li><a href="https://medium.com/follow-the-rabbit/has-duolingo-lost-its-streak-9cb362232eeb">Has Duolingo Lost Its Streak? - Matt Jones - Medium</a></li>
<li><a href="https://www.bbc.com/news/articles/c722gne7qngo">McDonald’s removes AI drive-throughs after order errors</a></li>
<li><a href="https://futurism.com/openai-use-cheating-homework">OpenAI Usage Plummets in the Summer, When Students Aren’t Cheating on Homework</a></li>
<li><a href="https://lithub.com/what-happened-when-i-tried-to-replace-myself-with-chatgpt-in-my-english-classroom/">What Happened When I Tried to Replace Myself with ChatGPT in My English Classroom - Literary Hub</a></li>
<li><a href="https://www.youtube.com/watch?app=desktop&v=CcTC3I5SF7k">Learning to code in the age of AI — Sheena O’Connell - YouTube</a></li>
<li><a href="https://blog.jetbrains.com/author/jodie-burchell-jetbrains-com/">Jodie Burchell - The JetBrains Blog</a></li>
<li><a href="https://t-redactyl.io/">Jodie Burchell’s Blog - Standard error</a></li>
<li><a href="https://bsky.app/profile/t-redactyl.bsky.social">Jodie Burchell (@t-redactyl.bsky.social) — Bluesky</a></li>
<li><a href="https://fosstodon.org/@t_redactyl">Jodie Burchell 🇦🇺🇩🇪 (@t_redactyl@fosstodon.org) - Fosstodon</a></li>
<li><a href="https://www.jetbrains.com/">JetBrains: Essential tools for software developers and teams</a></li>
</ul>
<p>Level up your Python skills with our expert-led courses:</p>
<ul>
<li><a href="https://realpython.com/courses/python-decorators-101/">Python Decorators 101</a></li>
<li><a href="https://realpython.com/courses/history-python-versions-features/">A History of Python Versions and Features</a></li>
<li><a href="https://realpython.com/courses/closures-examples-use-cases/">Exploring Python Closures: Examples and Use Cases</a></li>
</ul> <p><a rel="payment" href="https://realpython.com/join">Support the podcast & join our community of Pythonistas</a></p>
Exploring Modern Sentiment Analysis Approaches in Python
<p>What are the current approaches for analyzing emotions within a piece of text? Which tools and Python packages should you use for sentiment analysis? This week, Jodie Burchell, developer advocate for data science at JetBrains, returns to the show to discuss modern sentiment analysis in Python.</p>
<p>Jodie holds a PhD in clinical psychology. We discuss how her interest in studying emotions has continued throughout her career. </p>
<p>In this episode, Jodie covers three ways to approach sentiment analysis. We start by discussing traditional lexicon-based and machine-learning approaches. Then, we dive into how specific types of LLMs can be used for the task. We also share multiple resources so you can continue to explore sentiment analysis on your own.</p>
<p>This week’s episode is brought to you by Sentry.</p>
<div class="alert alert-primary" role="alert">
<p><strong>Course Spotlight:</strong> <a href="https://realpython.com/courses/text-classification-with-keras/">Learn Text Classification With Python and Keras</a></p>
<p>In this course, you’ll learn about Python text classification with Keras, working your way from a bag-of-words model with logistic regression to more advanced methods, such as convolutional neural networks. You’ll see how you can use pretrained word embeddings, and you’ll squeeze more performance out of your model through hyperparameter optimization.</p>
</div>
<p>Topics:</p>
<ul>
<li>00:00:00 – Introduction</li>
<li>00:02:31 – Conference talks in 2024</li>
<li>00:04:23 – Background on sentiment analysis and studying feelings</li>
<li>00:07:09 – What led you to study emotions?</li>
<li>00:08:57 – Dimensional emotion classification</li>
<li>00:10:42 – Different types of sentiment analysis</li>
<li>00:14:28 – Lexicon-based approaches</li>
<li>00:17:50 – VADER - Valence Aware Dictionary and sEntiment Reasoner</li>
<li>00:19:41 – TextBlob and subjectivity scoring </li>
<li>00:21:48 – Sponsor: Sentry</li>
<li>00:22:52 – Measuring sentiment of New Year’s resolutions</li>
<li>00:27:28 – Lexicon-based approaches links for experimenting</li>
<li>00:28:35 – Multiple language support in lexicon-based packages </li>
<li>00:35:23 – Machine learning techniques</li>
<li>00:39:20 – Tools for this approach</li>
<li>00:42:54 – Video Course Spotlight</li>
<li>00:44:15 – Advantages to the machine learning models approach </li>
<li>00:45:55 – Large language model approach</li>
<li>00:48:44 – Encoder vs decoder models</li>
<li>00:52:09 – Comparing the concept of fine-tuning</li>
<li>00:56:49 – Is this a recent development?</li>
<li>00:58:08 – Ways to practice with these techniques</li>
<li>01:00:10 – Do you find this to be a promising approach?</li>
<li>01:07:45 – Resources to practice with all the techniques</li>
<li>01:11:06 – Upcoming conference talks</li>
<li>01:11:56 – Thanks and goodbye</li>
</ul>
<p>Show Links:</p>
<ul>
<li><a href="https://blog.jetbrains.com/pycharm/2024/12/introduction-to-sentiment-analysis-in-python/">Introduction to Sentiment Analysis in Python - The PyCharm Blog</a></li>
<li><a href="https://blog.jetbrains.com/pycharm/2024/12/how-to-do-sentiment-analysis-with-large-language-models/">How to Do Sentiment Analysis With Large Language Models - The PyCharm Blog</a></li>
<li><a href="https://www.youtube.com/watch?v=innz9iBIAdU">Talks - Jodie Burchell: Lies, damned lies and large language models - YouTube</a></li>
<li><a href="https://www.youtube.com/watch?v=4jGwEOGunrU">Mirror, mirror: LLMs and the illusion of humanity - Jodie Burchell - YouTube</a></li>
<li><a href="https://www.youtube.com/watch?v=lLNJld729bc">Separating fact from fiction in a world of AI fairytales - Jodie Burchell - NDC London 2024 - YouTube</a></li>
<li><a href="https://www.youtube.com/watch?v=yjfSZu246zE">Hurt Feelings (Rap Version) - Flight Of The Conchords (Lyrics) - YouTube</a></li>
<li><a href="https://www.paulekman.com/universal-emotions/">Universal Emotions - What are Emotions? - Paul Ekman Group</a></li>
<li><a href="https://www.nltk.org/api/nltk.sentiment.vader.html">VADER - nltk.sentiment.vader module</a></li>
<li><a href="https://github.com/clips/pattern">clips/pattern: Web mining module for Python, with tools for scraping, natural language processing, machine learning</a></li>
<li><a href="https://textblob.readthedocs.io/en/dev/">TextBlob: Simplified Text Processing — TextBlob documentation</a></li>
<li><a href="https://www.goodreads.com/book/show/19795.Power_vs_Force">Power vs. Force: The Hidden Determinants of Human Behavior by David R. Hawkins - Goodreads</a></li>
<li><a href="https://realpython.com/podcasts/rpp/36/">Episode #36: Sentiment Analysis, Fourier Transforms, and More Python Data Science – The Real Python Podcast</a></li>
<li><a href="https://realpython.com/sentiment-analysis-python/">Use Sentiment Analysis With Python to Classify Movie Reviews – Real Python</a></li>
<li><a href="https://realpython.com/python-nltk-sentiment-analysis/">Sentiment Analysis: First Steps With Python’s NLTK Library – Real Python</a></li>
<li><a href="https://www.youtube.com/watch?v=c1zcPLVbhg4">Sentiment Analysis in DataSpell with @JetBrainsTV - YouTube</a></li>
<li><a href="https://realpython.com/podcasts/rpp/119/">Episode #119: Natural Language Processing and How ML Models Understand Text – The Real Python Podcast</a></li>
<li><a href="https://spacy.io/">spaCy - Industrial-strength Natural Language Processing in Python</a></li>
<li><a href="https://huggingface.co/datasets/fancyzhx/amazon_polarity">amazon_polarity - Datasets at Hugging Face</a></li>
<li><a href="https://blog.jetbrains.com/pycharm/2024/12/introduction-to-sentiment-analysis-in-python/">Introduction to Sentiment Analysis in Python - The PyCharm Blog</a></li>
<li><a href="https://www.kaggle.com/">Kaggle: Your Machine Learning and Data Science Community</a></li>
<li><a href="https://zsbit.pl/poprzednie-edycje/">ZS BIT</a></li>
<li><a href="https://www.aicommunityday.nl/">AI Community Day - 10 December 2024</a></li>
<li><a href="https://blog.jetbrains.com/author/jodie-burchell-jetbrains-com/">Jodie Burchell - The JetBrains Blog</a></li>
<li><a href="https://t-redactyl.io/">Jodie Burchell’s Blog - Standard error</a></li>
<li><a href="https://twitter.com/t_redactyl">Jodie Burchell 🇦🇺🇩🇪 (@t_redactyl) - Twitter</a></li>
<li><a href="https://bsky.app/profile/t-redactyl.bsky.social">Jodie Burchell (@t-redactyl.bsky.social) — Bluesky</a></li>
<li><a href="https://fosstodon.org/@t_redactyl">Jodie Burchell 🇦🇺🇩🇪 (@t_redactyl@fosstodon.org) - Fosstodon</a></li>
<li><a href="https://www.jetbrains.com/">JetBrains: Essential tools for software developers and teams</a></li>
</ul>
<p>Level up your Python skills with our expert-led courses:</p>
<ul>
<li><a href="https://realpython.com/courses/data-cleaning-with-pandas-and-numpy/">Data Cleaning With pandas and NumPy</a></li>
<li><a href="https://realpython.com/courses/text-classification-with-keras/">Learn Text Classification With Python and Keras</a></li>
<li><a href="https://realpython.com/courses/astrophysics-pandas-matplotlib/">Exploring Astrophysics in Python With pandas and Matplotlib</a></li>
</ul> <p><a rel="payment" href="https://realpython.com/join">Support the podcast & join our community of Pythonistas</a></p>
Measuring Bias, Toxicity, and Truthfulness in LLMs With Python
<p>How can you measure the quality of a large language model? What tools can measure bias, toxicity, and truthfulness levels in a model using Python? This week on the show, Jodie Burchell, developer advocate for data science at JetBrains, returns to discuss techniques and tools for evaluating LLMs With Python.</p>
<p>Jodie provides some background on large language models and how they can absorb vast amounts of information about the relationship between words using a type of neural network called a transformer. We discuss training datasets and the potential quality issues with crawling uncurated sources. </p>
<p>We dig into ways to measure levels of bias, toxicity, and hallucinations using Python. Jodie shares three benchmarking datasets and links to resources to get you started. We also discuss ways to augment models using agents or plugins, which can access search engine results or other authoritative sources.</p>
<p>This week’s episode is brought to you by Intel.</p>
<div class="alert alert-primary" role="alert">
<p><strong>Course Spotlight:</strong> <a href="https://realpython.com/courses/text-classification-with-keras/">Learn Text Classification With Python and Keras</a></p>
<p>In this course, you’ll learn about Python text classification with Keras, working your way from a bag-of-words model with logistic regression to more advanced methods, such as convolutional neural networks. You’ll see how you can use pretrained word embeddings, and you’ll squeeze more performance out of your model through hyperparameter optimization.</p>
</div>
<p>Topics:</p>
<ul>
<li>00:00:00 – Introduction</li>
<li>00:02:19 – Testing characteristics of LLMs with Python</li>
<li>00:04:18 – Background on LLMs</li>
<li>00:08:35 – Training of models</li>
<li>00:14:23 – Uncurated sources of training</li>
<li>00:16:12 – Safeguards and prompt engineering</li>
<li>00:21:19 – TruthfulQA and creating a more strict prompt</li>
<li>00:23:20 – Information that is out of date</li>
<li>00:26:07 – WinoBias for evaluating gender stereotypes</li>
<li>00:28:30 – BOLD dataset for evaluating bias</li>
<li>00:30:28 – Sponsor: Intel</li>
<li>00:31:18 – Using Hugging Face to start testing with Python</li>
<li>00:35:25 – Using the transformers package</li>
<li>00:37:34 – Using langchain for proprietary models</li>
<li>00:43:04 – Putting the tools together and evaluating</li>
<li>00:47:19 – Video Course Spotlight</li>
<li>00:48:29 – Assessing toxicity</li>
<li>00:50:21 – Measuring bias</li>
<li>00:54:40 – Checking the hallucination rate</li>
<li>00:56:22 – LLM leaderboards</li>
<li>00:58:17 – What helped ChatGPT leap forward?</li>
<li>01:06:01 – Improvements of what is being crawled</li>
<li>01:07:32 – Revisiting agents and RAG</li>
<li>01:11:03 – ChatGPT plugins and Wolfram-Alpha</li>
<li>01:13:06 – How can people follow your work online?</li>
<li>01:14:33 – Thanks and goodbye</li>
</ul>
<p>Background Links:</p>
<ul>
<li><a href="https://builtin.com/data-science/beginners-guide-language-models">A Beginner’s Guide to Language Models - Built In</a></li>
<li><a href="https://www.youtube.com/watch?v=NpmnWgQgcsA">ChatGPT - Explained! - YouTube</a></li>
</ul>
<p>Dataset Links:</p>
<ul>
<li><a href="https://huggingface.co/datasets/truthful_qa?row=4">truthful_qa - Datasets at Hugging Face</a></li>
<li><a href="https://huggingface.co/datasets/wino_bias">wino_bias - Datasets at Hugging Face</a></li>
<li><a href="https://huggingface.co/datasets/AlexaAI/bold">bold - Datasets at Hugging Face</a></li>
</ul>
<p>Tutorials and Documentation for Python Packages:</p>
<ul>
<li><a href="https://huggingface.co/blog/evaluating-llm-bias">Evaluating Language Model Bias with 🤗 Evaluate</a></li>
<li><a href="https://colab.research.google.com/drive/1-HDJUcPMKEF-E7Hapih0OmA1xTW2hdAv#scrollTo=yX8ciyVWKiuO">Hugging Face - HF_bias_evaluation - Google Colab</a></li>
<li><a href="https://huggingface.co/docs/datasets/loading">General Usage - Load a Dataset - Hugging Face</a></li>
<li><a href="https://huggingface.co/tasks/text-generation">What is Text Generation? - Hugging Face</a></li>
<li><a href="https://huggingface.co/docs/evaluate/index">🤗 Evaluate - Library Evaluating ML Models</a></li>
<li><a href="https://python.langchain.com/docs/get_started/quickstart">Python Quickstart - 🦜️🔗 Langchain</a></li>
</ul>
<p>Measurement Links:</p>
<ul>
<li><a href="https://huggingface.co/spaces/evaluate-measurement/toxicity">Toxicity - a Hugging Face Space by evaluate-measurement</a></li>
<li><a href="https://huggingface.co/spaces/evaluate-measurement/regard">Regard - a Hugging Face Space by evaluate-measurement</a></li>
<li><a href="https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard">Open LLM Leaderboard - a Hugging Face Space</a></li>
</ul>
<p>Training Data for LLMs:</p>
<ul>
<li><a href="https://commoncrawl.org/">Common Crawl - Open Repository of Web Crawl Data</a></li>
<li><a href="https://pile.eleuther.ai/">The Pile</a></li>
<li><a href="https://arxiv.org/abs/2306.01116">The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora</a></li>
</ul>
<p>Agents and Plugin Links:</p>
<ul>
<li><a href="https://huggingface.co/docs/transformers/transformers_agents">Transformers Agents - Hugging Face</a></li>
<li><a href="https://python.langchain.com/docs/modules/agents/">Agents - 🦜️🔗 Langchain</a></li>
<li><a href="https://writings.stephenwolfram.com/2023/03/chatgpt-gets-its-wolfram-superpowers/">ChatGPT Gets Its “Wolfram Superpowers”! - Stephen Wolfram</a></li>
</ul>
<p>Additional Links:</p>
<ul>
<li><a href="https://www.theverge.com/features/23764584/ai-artificial-intelligence-data-notation-labor-scale-surge-remotasks-openai-chatbots">Inside the AI Factory: The Humans that Make Tech Seem Human - The Verge</a></li>
<li><a href="https://blog.jetbrains.com/author/jodie-burchell-jetbrains-com/">Jodie Burchell - The JetBrains Blog</a></li>
<li><a href="https://t-redactyl.io/">Jodie Burchell’s Blog - Standard error</a></li>
<li><a href="https://twitter.com/t_redactyl">Jodie Burchell 🇦🇺🇩🇪 (@t_redactyl) - Twitter</a></li>
<li><a href="https://fosstodon.org/@t_redactyl#">Jodie Burchell 🇦🇺🇩🇪 (@t_redactyl@fosstodon.org) - Fosstodon</a></li>
<li><a href="https://www.jetbrains.com/">JetBrains: Essential tools for software developers and teams</a></li>
</ul>
<p>Level up your Python skills with our expert-led courses:</p>
<ul>
<li><a href="https://realpython.com/courses/data-cleaning-with-pandas-and-numpy/">Data Cleaning With pandas and NumPy</a></li>
<li><a href="https://realpython.com/courses/python-folium-web-maps-from-data/">Creating Web Maps From Your Data With Python Folium</a></li>
<li><a href="https://realpython.com/courses/text-classification-with-keras/">Learn Text Classification With Python and Keras</a></li>
</ul> <p><a rel="payment" href="https://realpython.com/join">Support the podcast & join our community of Pythonistas</a></p>
Using NumPy and Linear Algebra for Faster Python Code
<p>Are you still using loops and lists to process your data in Python? Have you heard of a Python library with optimized data structures and built-in operations that can speed up your data science code? This week on the show, Jodie Burchell, developer advocate for data science at JetBrains, returns to share secrets for harnessing linear algebra and NumPy for your projects.</p>
<p>Jodie details how most people begin their data science journey using loops to iterate over values and apply operations sequentially. We talk about how loops are friendly for beginners, being clear to read and easy to debug, but unfortunately don’t scale well, especially with large amounts of data.</p>
<p>Jodie shares some of the basics of linear algebra and how to organize data into vectors. We talk about how the NumPy library leverages those concepts to improve data processing. We discuss how the library includes operations for vector and matrix addition and subtraction, and why these operations are more efficient than loops. We also cover how NumPy stores arrays in memory and when working with them is faster vs when it’s not.</p>
<div class="alert alert-primary" role="alert">
<p><strong>Course Spotlight:</strong> <a href="https://realpython.com/courses/data-cleaning-with-pandas-and-numpy/">Data Cleaning With pandas and NumPy</a></p>
<p>In this video course, you’ll learn how to clean up messy data using pandas and NumPy. You’ll become equipped to deal with a range of problems, such as missing values, inconsistent formatting, malformed records, and nonsensical outliers.</p>
</div>
<p>Topics:</p>
<ul>
<li>00:00:00 – Introduction </li>
<li>00:02:35 – Vectorize all the things! - PyCon UK 2022 Talk</li>
<li>00:06:39 – Becoming familiar with linear algebra</li>
<li>00:09:05 – Beginners start with loops</li>
<li>00:11:25 – Starting with basic linear algebra</li>
<li>00:12:25 – The basic unit of a vector</li>
<li>00:18:06 – NumPy representing vectors in Python</li>
<li>00:23:25 – Sponsor: InfluxDB</li>
<li>00:24:13 – Block management</li>
<li>00:25:54 – Replacing a loop with vector-based operations</li>
<li>00:34:06 – NumPy broadcasting</li>
<li>00:38:52 – Approximating nearest neighbors</li>
<li>00:43:49 – Video Course Spotlight</li>
<li>00:45:15 – Solving the problem</li>
<li>00:46:44 – Getting rid of nested loops</li>
<li>00:48:54 – A peek under the hood</li>
<li>00:53:28 – How arrays vs lists are stored in memory</li>
<li>01:00:24 – Considering a GPU</li>
<li>01:03:37 – Real Python resources on the subject</li>
<li>01:04:08 – Upcoming talks and conferences</li>
<li>01:07:31 – Thanks and goodbye</li>
</ul>
<p>Show Links:</p>
<ul>
<li><a href="https://www.youtube.com/watch?v=9_mhjjlKjDo">Vectorize all the things! How basic linear algebra can speed up your data science code - YouTube</a></li>
<li><a href="https://math.mit.edu/~gs/linearalgebra/ila5/indexila5.html">Introduction to Linear Algebra, 5th Edition</a></li>
<li><a href="https://ocw.mit.edu/courses/18-06-linear-algebra-spring-2010/">Linear Algebra - Mathematics - MIT OpenCourseWare</a></li>
<li><a href="https://math.mit.edu/~gs/learningfromdata/">Linear Algebra and Learning from Data</a></li>
<li><a href="https://realpython.com/python-linear-algebra/">Linear Algebra in Python: Matrix Inverses and Least Squares</a></li>
<li><a href="https://numpy.org/doc/stable/user/absolute_beginners.html">NumPy: the absolute basics for beginners - NumPy Manual</a></li>
<li><a href="https://numpy.org/doc/stable/user/basics.broadcasting.html">Broadcasting — NumPy v1.24 Manual</a></li>
<li><a href="https://github.com/spotify/annoy">spotify/annoy: Approximate Nearest Neighbors in C++/Python optimized</a></li>
<li><a href="https://realpython.com/numpy-array-programming/">Look Ma, No For-Loops: Array Programming With NumPy – Real Python</a></li>
<li><a href="https://realpython.com/numpy-tutorial/">NumPy Tutorial: Your First Steps Into Data Science in Python – Real Python</a></li>
<li><a href="https://realpython.com/pandas-iterate-over-rows/">How to Iterate Over Rows in pandas, and Why You Shouldn’t – Real Python</a></li>
<li><a href="https://www.datacamp.com/radar-2023">RADAR: Thrive in the era of data - DataCamp</a></li>
<li><a href="https://2023.pythonwebconf.com/presentations/vectorize-all-the-things-using-linear-algebra-and-numpy-to-make-your-python-code-lightning-fast">Vectorize all the things! Using linear algebra and NumPy to make your Python code lightning fast. - Python Web Conference 2023</a></li>
<li><a href="https://us.pycon.org/2023/schedule/presentation/140/">Jodie Burchell - PyCon US 2023</a></li>
<li><a href="https://t-redactyl.io/">Jodie Burchell’s Blog - Standard error</a></li>
<li><a href="https://twitter.com/t_redactyl">Jodie Burchell 🇦🇺🇩🇪 (@t_redactyl) - Twitter</a></li>
<li><a href="https://fosstodon.org/@t_redactyl#">Jodie Burchell 🇦🇺🇩🇪 (@t_redactyl@fosstodon.org) - Fosstodon</a></li>
<li><a href="https://www.jetbrains.com/">JetBrains: Essential tools for software developers and teams</a></li>
</ul>
<p>Level up your Python skills with our expert-led courses:</p>
<ul>
<li><a href="https://realpython.com/courses/data-cleaning-with-pandas-and-numpy/">Data Cleaning With pandas and NumPy</a></li>
<li><a href="https://realpython.com/courses/python-histograms/">Histogram Plotting in Python: NumPy, Matplotlib, Pandas & Seaborn</a></li>
<li><a href="https://realpython.com/courses/numpy-arange/">Using NumPy's np.arange() Effectively</a></li>
</ul> <p><a rel="payment" href="https://realpython.com/join">Support the podcast & join our community of Pythonistas</a></p>
Preparing Data to Measure True Machine Learning Model Performance
<p>How do you prepare a dataset for machine learning (ML)? How do you go beyond cleaning the data and move toward measuring how the model performs? This week on the show, Jodie Burchell, developer advocate for data science at JetBrains, returns to talk about strategies for better ML model performance.</p>
<p>Jodie starts by defining some terms for the conversation. We talk about targets, features, and supervised learning.</p>
<p>We discuss three common ways that data can alter model performance and which Python tools can help spot and avoid them. Jodie shares personal experiences of working through these pitfalls. We also share a healthy collection of resources to explore and learn more.</p>
<div class="alert alert-primary" role="alert">
<p><strong>Course Spotlight:</strong> <a href="https://realpython.com/courses/data-pandas-concat-and-merge/">Combining Data in pandas With concat() and merge()</a></p>
<p>In this video course, you’ll learn two techniques for combining data in pandas: merge() and concat(). Combining Series and DataFrame objects in pandas is a powerful way to gain new insights into your data. </p>
</div>
<p>Topics:</p>
<ul>
<li>00:00:00 – Introduction</li>
<li>00:01:46 – Recent conference talks</li>
<li>00:03:24 – How to prepare your data for model performance</li>
<li>00:04:24 – Vocabulary: target, features, and supervised learning</li>
<li>00:06:28 – The curse of dimensionality</li>
<li>00:08:57 – Overfitting</li>
<li>00:11:08 – Underfitting</li>
<li>00:12:11 – Splitting the dataset</li>
<li>00:13:39 – K-fold cross validation</li>
<li>00:18:30 – Data leakage</li>
<li>00:21:36 – Checking for duplicates</li>
<li>00:26:23 – Applying transformations only after splitting data</li>
<li>00:31:16 – Imbalanced data</li>
<li>00:36:36 – Using ML to balance data</li>
<li>00:41:05 – Informing your model of the imbalance</li>
<li>00:42:56 – Video Course Spotlight</li>
<li>00:44:20 – Accuracy used as a measure</li>
<li>00:49:05 – Scikit-learn method <code>classification_table</code></li>
<li>00:50:43 – Jet Brains blog post and conference talk</li>
<li>00:52:18 – How can people follow your work online?</li>
<li>00:54:39 – Upcoming webinars</li>
<li>00:56:20 – Thanks and goodbye</li>
</ul>
<p>Show Links:</p>
<ul>
<li><a href="https://blog.jetbrains.com/datalore/2022/11/08/how-to-prepare-your-dataset-for-machine-learning-and-analysis/">How to Prepare Your Dataset for Machine Learning and Analysis - The JetBrains Datalore Blog</a></li>
<li><a href="https://en.wikipedia.org/wiki/Curse_of_dimensionality">Curse of dimensionality - Wikipedia</a></li>
<li><a href="https://towardsdatascience.com/overfitting-vs-underfitting-a-complete-example-d05dd7e19765">Overfitting vs. Underfitting: A Complete Example - Will Koehrsen</a></li>
<li><a href="https://machinelearningmastery.com/k-fold-cross-validation/">A Gentle Introduction to k-fold Cross-Validation - MachineLearningMastery.com</a></li>
<li><a href="https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html">sklearn.model_selection.train_test_split — scikit-learn documentation</a></li>
<li><a href="https://scikit-learn.org/stable/modules/cross_validation.html">Cross-validation: evaluating estimator performance — scikit-learn documentation</a></li>
<li><a href="https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.cross_val_score.html#sklearn.model_selection.cross_val_score">sklearn.model_selection.cross_val_score — scikit-learn documentation</a></li>
<li><a href="https://www.analyticsvidhya.com/blog/2021/07/data-leakage-and-its-effect-on-the-performance-of-an-ml-model/">Data Leakage And Its Effect On The Performance of An ML Model</a></li>
<li><a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.duplicated.html">pandas.DataFrame.duplicated — pandas documentation</a></li>
<li><a href="https://realpython.com/pandas-groupby/">pandas GroupBy: Your Guide to Grouping Data in Python – Real Python</a></li>
<li><a href="https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.groupby.html">pandas.DataFrame.groupby — pandas documentation</a></li>
<li><a href="https://medium.com/nerd-for-tech/difference-fit-transform-and-fit-transform-method-in-scikit-learn-b0a4efcab804">Difference between fit(), transform() and fit_transform() method in Scikit-learn - Aishwarya Chand: Nerd For Tech</a></li>
<li><a href="https://developers.google.com/machine-learning/data-prep/construct/sampling-splitting/imbalanced-data">Imbalanced Data in Machine Learning - Google Developers</a></li>
<li><a href="https://imbalanced-learn.org/stable/under_sampling.html">Under-sampling — imbalanced-learn.org</a></li>
<li><a href="https://imbalanced-learn.org/stable/over_sampling.html">Over-sampling — imbalanced-learn.org</a></li>
<li><a href="https://gretel.ai/learn/getting-started">Learn - Getting Started with Gretel.ai</a></li>
<li><a href="https://www.tensorflow.org/tutorials/structured_data/imbalanced_data#class_weights">Classification on imbalanced data: Class weights - TensorFlow Core</a></li>
<li><a href="https://machinelearningmastery.com/tour-of-evaluation-metrics-for-imbalanced-classification/">Tour of Evaluation Metrics for Imbalanced Classification - MachineLearningMastery.com</a></li>
<li><a href="https://www.cloudbrew.be/">CloudBrew - A two-day conference by AZUG, the Belgium Microsoft Azure User Group</a></li>
<li><a href="https://t-redactyl.io/">Jodie Burchell’s Blog - Standard error</a></li>
<li><a href="https://twitter.com/t_redactyl">Jodie Burchell 🇦🇺🇩🇪 (@t_redactyl) - Twitter</a></li>
<li><a href="https://fosstodon.org/@t_redactyl#">Jodie Burchell 🇦🇺🇩🇪 (@t_redactyl@fosstodon.org) - Fosstodon</a></li>
<li><a href="https://www.jetbrains.com/">JetBrains: Essential tools for software developers and teams</a></li>
</ul>
<p>Level up your Python skills with our expert-led courses:</p>
<ul>
<li><a href="https://realpython.com/courses/data-cleaning-with-pandas-and-numpy/">Data Cleaning With pandas and NumPy</a></li>
<li><a href="https://realpython.com/courses/rest-apis-with-django-ninja/">Sneaky REST APIs With Django Ninja</a></li>
<li><a href="https://realpython.com/courses/data-pandas-concat-and-merge/">Combining Data in pandas With concat() and merge()</a></li>
</ul> <p><a rel="payment" href="https://realpython.com/join">Support the podcast & join our community of Pythonistas</a></p>
629: Software for Efficient Data Science
Has the term developer advocacy ever left you scratching your head? This week data science developer advocate for JetBrains, Dr. Jodie Burchell, joins Jon Krohn to shed light on her responsibilities and why it's a role you might want to consider. Jodie also dives into building reproducible data science workflows and the keys to working effectively with real-world data.
This episode is brought to you by Iterative (iterative.ai), the open-source company behind DVC. Interested in sponsoring a SuperDataScience Podcast episode? Visit JonKrohn.com/podcast for sponsorship information.
In this episode you will learn:
• Jodie’s background in psychology [2:22]
• Jodie's tips for real-world data preparation [6:55]
• Tour JetBrains' developer tools: PyCharm, DataSpell and Datalore [10:41]
• What is a data science developer advocate? [38:47]
• The books that Jodie's co-authored [46:18]
• Jodie's favorite Python libraries [58:33]
• How to have reproducible data science workflows [1:01:36]
Additional materials: www.superdatascience.com/629
Moving NLP Forward With Transformer Models and Attention
<p>What’s the big breakthrough for Natural Language Processing (NLP) that has dramatically advanced machine learning into deep learning? What makes these transformer models unique, and what defines “attention?” This week on the show, Jodie Burchell, developer advocate for data science at JetBrains, continues our talk about how machine learning (ML) models understand and generate text.</p>
<p>This episode is a continuation of the conversation in episode #119. Jodie builds on the concepts of bag-of-words, word2vec, and simple embedding models. We talk about the breakthrough mechanism called “attention,” which allows for parallelization in building models. </p>
<p>We also discuss the two major transformer models, BERT and GPT3. Jodie continues to share multiple resources to help you continue exploring modeling and NLP with Python.</p>
<div class="alert alert-primary" role="alert">
<p><strong>Course Spotlight:</strong> <a href="https://realpython.com/courses/build-neural-network-python-ai/">Building a Neural Network & Making Predictions With Python AI</a></p>
<p>In this step-by-step course, you’ll build a neural network from scratch as an introduction to the world of artificial intelligence (AI) in Python. You’ll learn how to train your neural network and make predictions based on a given dataset.</p>
</div>
<p>Topics:</p>
<ul>
<li>00:00:00 – Introduction</li>
<li>00:02:20 – Where we left off with word2vec…</li>
<li>00:03:35 – Example of losing context</li>
<li>00:06:50 – Working at scale and adding attention</li>
<li>00:12:34 – Multiple levels of training for the model </li>
<li>00:14:10 – Attention is the basis for transformer models</li>
<li>00:15:07 – BERT (Bidirectional Encoder Representations from Transformers)</li>
<li>00:16:29 – GPT (Generative Pre-trained Transformer)</li>
<li>00:19:08 – Video Course Spotlight</li>
<li>00:20:08 – How far have we moved forward?</li>
<li>00:20:41 – Access to GPT-2 via Hugging Face</li>
<li>00:23:56 – How to access and use these models?</li>
<li>00:30:42 – Cost of training GPT-3</li>
<li>00:35:01 – Resources to practice and learn with BERT</li>
<li>00:38:19 – GPT-3 and GitHub Copilot</li>
<li>00:44:35 – DALL-E is a transformer</li>
<li>00:46:13 – Help yourself to the show notes!</li>
<li>00:49:19 – How can people follow your work?</li>
<li>00:50:03 – Thanks and goodbye</li>
</ul>
<p>Show Links:</p>
<ul>
<li><a href="https://en.wikipedia.org/wiki/Recurrent_neural_network">Recurrent neural network - Wikipedia</a></li>
<li><a href="https://en.wikipedia.org/wiki/Long_short-term_memory">Long short-term memory - Wikipedia</a></li>
<li><a href="https://en.wikipedia.org/wiki/Vanishing_gradient_problem">Vanishing gradient problem - Wikipedia</a></li>
<li><a href="https://www.mygreatlearning.com/blog/the-vanishing-gradient-problem/">Vanishing Gradient Problem | What is Vanishing Gradient Problem?</a></li>
<li><a href="https://arxiv.org/abs/1706.03762">Attention Is All You Need | Cornell University</a></li>
<li><a href="https://jalammar.github.io/visualizing-neural-machine-translation-mechanics-of-seq2seq-models-with-attention/">Visualizing A Neural Machine Translation Model (Mechanics of Seq2seq Models With Attention) – Jay Alammar</a></li>
<li><a href="https://arxiv.org/abs/2204.10019">Standing on the Shoulders of Giant Frozen Language Models | Cornell University</a></li>
<li><a href="https://www.youtube.com/watch?v=NKAK8ttzd6Q">#datalift22 Embeddings paradigm shift: Model training to vector similarity search by Nava Levy - YouTube</a></li>
<li><a href="https://www.youtube.com/watch?v=TQQlZhbC5ps&list=LL&index=4">Transformer Neural Networks - EXPLAINED! (Attention is all you need) - YouTube</a></li>
<li><a href="https://huggingface.co/blog/bert-101">BERT 101 - State Of The Art NLP Model Explained</a></li>
<li><a href="https://www.youtube.com/watch?v=MQnJZuBGmSQ">How GPT3 Works - Easily Explained with Animations - YouTube</a></li>
<li><a href="https://transformer.huggingface.co/doc/gpt2-large">Write With Transformer (GPT2 Live Playground Tool) - Hugging Face</a></li>
<li><a href="https://opt.alpa.ai/">Language Model with Alpa (GPT3 Live Playground Tool) OPT-175B</a></li>
<li><a href="https://www.bigdata.fm/">Big Data | Music</a></li>
<li><a href="https://beta.openai.com/playground">OpenAI API</a></li>
<li><a href="https://huggingface.co/docs/transformers/notebooks">🤗 (Hugging Face)Transformers Notebooks</a></li>
<li><a href="https://realpython.com/github-copilot-python/">GitHub Copilot: Fly With Python at the Speed of Thought</a></li>
<li><a href="https://twitter.com/sotak/status/1546452033816985601">GitHub Copilot learned about the daily struggle of JavaScript developers after being trained on billions of lines of code. | Marek Sotak on Twitter</a></li>
<li><a href="https://t-redactyl.io/">Jodie Burchell’s Blog - Standard error</a></li>
<li><a href="https://twitter.com/t_redactyl">Jodie Burchell 🇦🇺🇩🇪 (@t_redactyl) / Twitter</a></li>
<li><a href="https://www.jetbrains.com/">JetBrains: Essential tools for software developers and teams</a></li>
</ul>
<p>Level up your Python skills with our expert-led courses:</p>
<ul>
<li><a href="https://realpython.com/courses/build-neural-network-python-ai/">Building a Neural Network & Making Predictions With Python AI</a></li>
<li><a href="https://realpython.com/courses/text-classification-with-keras/">Learn Text Classification With Python and Keras</a></li>
<li><a href="https://realpython.com/courses/pandas-dataframe-working-with-data/">The pandas DataFrame: Working With Data Efficiently</a></li>
</ul> <p><a rel="payment" href="https://realpython.com/join">Support the podcast & join our community of Pythonistas</a></p>
Natural Language Processing and How ML Models Understand Text
<p>How do you process and classify text documents in Python? What are the fundamental techniques and building blocks for Natural Language Processing (NLP)? This week on the show, Jodie Burchell, developer advocate for data science at JetBrains, talks about how machine learning (ML) models understand text.</p>
<p>Jodie explains how ML models require data in a structured format, which involves transforming text documents into columns and rows. She covers the most straightforward approach, called binary vectorization. We discuss the bag-of-words method and the tools of stemming, lemmatization, and count vectorization.</p>
<p>We jump into word embedding models next. Jodie talks about WordNet, Natural Language Toolkit (NLTK), word2vec, and Gensim. Our conversation lays a foundation for starting with text classification, implementing sentiment analysis, and building projects using these tools. Jodie also shares multiple resources to help you continue exploring NLP and modeling.</p>
<div class="alert alert-primary" role="alert">
<p><strong>Course Spotlight:</strong> <a href="https://realpython.com/courses/text-classification-with-keras/">Learn Text Classification With Python and Keras</a></p>
<p>In this course, you’ll learn about Python text classification with Keras, working your way from a bag-of-words model with logistic regression to more advanced methods, such as convolutional neural networks. You’ll see how you can use pretrained word embeddings, and you’ll squeeze more performance out of your model through hyperparameter optimization.</p>
</div>
<p>Topics:</p>
<ul>
<li>00:00:00 – Introduction</li>
<li>00:02:47 – Exploring the topic</li>
<li>00:06:00 – Perceived sentience of LaMDA </li>
<li>00:10:24 – How do we get started?</li>
<li>00:11:16 – What are classification and sentiment analysis?</li>
<li>00:13:03 – Transforming text in rows and columns</li>
<li>00:14:47 – Sponsor: Snyk</li>
<li>00:15:27 – Bag-of-words approach</li>
<li>00:19:12 – Stemming and lemmatization</li>
<li>00:22:05 – Capturing N-grams</li>
<li>00:25:34 – Count vectorization</li>
<li>00:27:14 – Stop words</li>
<li>00:28:46 – Text Frequency / Inverse Document Frequency (TFIDF) vectorization</li>
<li>00:32:28 – Potential projects for bag-of-words techniques</li>
<li>00:34:07 – Video Course Spotlight</li>
<li>00:35:20 – WordNet and NLTK package</li>
<li>00:37:27 – Word embeddings and word2vec</li>
<li>00:45:30 – Previous training and too many dimensions</li>
<li>00:50:07 – How to use word2vec and Gensim?</li>
<li>00:51:26 – What types of projects for word2vec and Gensim?</li>
<li>00:54:41 – Getting into GPT and BERT in another episode</li>
<li>00:56:11 – How to follow Jodie’s work?</li>
<li>00:57:36 – Thanks and goodbye</li>
</ul>
<p>Show Links:</p>
<ul>
<li><a href="https://slate.com/technology/2022/06/google-ai-sentience-lamda.html">Why Google’s “sentient” AI LaMDA is nothing like a person.</a></li>
<li><a href="https://medium.com/@emilymenonbender/on-nyt-magazine-on-ai-resist-the-urge-to-be-impressed-3d92fd9a0edd">On NYT Magazine on AI: Resist the Urge to be Impressed | Emily M. Bender | Medium</a> </li>
<li><a href="https://en.wikipedia.org/wiki/ELIZA">ELIZA - Wikipedia</a></li>
<li><a href="http://dhconnelly.com/paip-python/docs/paip/eliza.html">eliza.py - Python 2 version by Daniel Connelly</a></li>
<li><a href="https://github.com/dabraude/Pyliza">dabraude/Pyliza: Python3 Implementation of Eliza</a></li>
<li><a href="https://magneticpoetry.com/">magneticpoetry.com</a></li>
<li><a href="https://realpython.com/nltk-nlp-python/">Natural Language Processing With Python’s NLTK Package – Real Python</a></li>
<li><a href="https://realpython.com/python-keras-text-classification/">Practical Text Classification With Python and Keras – Real Python</a></li>
<li><a href="https://realpython.com/python-nltk-sentiment-analysis/">Sentiment Analysis: First Steps With Python’s NLTK Library – Real Python</a></li>
<li><a href="https://www.nltk.org/">NLTK: Natural Language Toolkit</a></li>
<li><a href="https://spacy.io/">spaCy · Industrial-strength Natural Language Processing in Python</a></li>
<li><a href="https://realpython.com/natural-language-processing-spacy-python/">Natural Language Processing With spaCy in Python - Real Python</a></li>
<li><a href="https://en.wikipedia.org/wiki/Stemming">Stemming - Wikipedia</a></li>
<li><a href="https://en.wikipedia.org/wiki/Lemmatisation">Lemmatization - Wikipedia</a></li>
<li><a href="https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html">Binary/Count Vectorization: sklearn.feature_extraction.text.CountVectorizer— scikit-learn</a></li>
<li><a href="https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html">TFIDF: sklearn.feature_extraction.text.TfidfVectorizer — scikit-learn</a></li>
<li><a href="https://www.nltk.org/api/nltk.stem.porter.html">Porter Stemmer: nltk.stem.porter module — NLTK</a></li>
<li><a href="https://www.nltk.org/api/nltk.stem.snowball.html">Snowball Stemmer: nltk.stem.snowball module — NLTK</a></li>
<li><a href="https://www.nltk.org/api/nltk.stem.wordnet.html">WordNet Lemmatizer: nltk.stem.wordnet module — NLTK</a></li>
<li><a href="https://spacy.io/api/lemmatizer">Lemmatizer · spaCy API Documentation</a></li>
<li><a href="https://elvinouyang.github.io/project/reuters-w2v-bow-get-started/">Applying Bag of Words and Word2Vec models on Reuters-21578 Dataset Elvin Ouyang’s Blog</a></li>
<li><a href="https://archive.ics.uci.edu/ml/datasets/Reuters-21578+Text+Categorization+Collection">UCI Machine Learning Repository: Reuters-21578 Text Categorization Collection Data Set</a> </li>
<li><a href="https://jalammar.github.io/illustrated-word2vec/">The Illustrated Word2vec – Jay Alammar</a></li>
<li><a href="https://analyticsindiamag.com/a-complete-guide-to-using-wordnet-in-nlp-applications/">A Complete Guide to Using WordNET in NLP Applications</a></li>
<li><a href="https://radimrehurek.com/gensim/">Gensim: Topic modeling for humans</a></li>
<li><a href="https://radimrehurek.com/gensim/auto_examples/index.html#tutorials-learning-oriented-lessons">Core Tutorials — gensim</a></li>
<li><a href="https://www.kaggle.com/datasets">Find Open Datasets and Machine Learning Projects | Kaggle</a></li>
<li><a href="https://www.youtube.com/watch?v=Xvg61XFkwoU">Engineering All Hands: Vectorise all the things! - YouTube</a></li>
<li><a href="https://2022.pycon.pt/">PyCon Portugal 2022</a></li>
<li><a href="https://ndcoslo.com/">NDC Oslo 2022 | Conference for Software Developers</a></li>
<li><a href="https://t-redactyl.io/">Jodie Burchell’s Blog - Standard error</a></li>
<li><a href="https://twitter.com/t_redactyl">Jodie Burchell 🇦🇺🇩🇪 (@t_redactyl) / Twitter</a></li>
<li><a href="https://www.jetbrains.com/">JetBrains: Essential tools for software developers and teams</a></li>
</ul>
<p>Level up your Python skills with our expert-led courses:</p>
<ul>
<li><a href="https://realpython.com/courses/data-cleaning-with-pandas-and-numpy/">Data Cleaning With pandas and NumPy</a></li>
<li><a href="https://realpython.com/courses/reading-writing-files-pandas/">Reading and Writing Files With pandas</a></li>
<li><a href="https://realpython.com/courses/text-classification-with-keras/">Learn Text Classification With Python and Keras</a></li>
</ul> <p><a rel="payment" href="https://realpython.com/join">Support the podcast & join our community of Pythonistas</a></p>