Reinventing the Python Notebook with Akshay Agrawal
Interactive notebooks were popularized by the Jupyter project and have since become a core tool for data science, research, and data exploration. However, traditional, imperative notebooks often break down as projects grow more complex. Hidden state, non-reproducible execution, poor version control ergonomics, and difficulty reusing notebook code in real software systems make it hard to move from exploration to production. At the same time, sharing results often requires collaborators to recreate entire environments, limiting interactivity and slowing feedback.
Marimo is an open-source, next-generation Python notebook designed to address these problems directly. Akshay Agrawal is the creator of Marimo and he previously worked at Google Brain. He joins the show with Kevin Ball to discuss the limitations of traditional notebooks, the design of reactive notebooks in Python, how marimo bridges research and production, and where notebooks fit in an increasingly agentic, AI-assisted development world.
Kevin Ball or KBall, is the vice president of engineering at Mento and an independent coach for engineers and engineering leaders. He co-founded and served as CTO for two companies, founded the San Diego JavaScript meetup, and organizes the AI inaction discussion group through Latent Space.
Please click here to see the transcript of this episode.
Sponsorship inquiries: sponsor@softwareengineeringdaily.com
The post Reinventing the Python Notebook with Akshay Agrawal appeared first on Software Engineering Daily.
911: The Future of Python Notebooks is Here, with Marimo’s Dr. Akshay Agrawal
Reproducibility, Python notebooks, and data science communities: Software developer Akshay Agrawal speaks to Jon Krohn about Marimo, the next-generation computational notebook for Python, how he built and fostered a thriving community around the product, and what makes this notebook so versatile and accessible for users.
Additional materials: www.superdatascience.com/911
This episode is brought to you by Trainium2, the latest AI chip from AWS and by the Dell AI Factory with NVIDIA.
Interested in sponsoring a SuperDataScience Podcast episode? Email natalie@superdatascience.com for sponsorship information.
Revolutionizing Python Notebooks with Marimo
Summary
In this episode of the Data Engineering Podcast Akshay Agrawal from Marimo discusses the innovative new Python notebook environment, which offers a reactive execution model, full Python integration, and built-in UI elements to enhance the interactive computing experience. He discusses the challenges of traditional Jupyter notebooks, such as hidden states and lack of interactivity, and how Marimo addresses these issues with features like reactive execution and Python-native file formats. Akshay also explores the broader landscape of programmatic notebooks, comparing Marimo to other tools like Jupyter, Streamlit, and Hex, highlighting its unique approach to creating data apps directly from notebooks and eliminating the need for separate app development. The conversation delves into the technical architecture of Marimo, its community-driven development, and future plans, including a commercial offering and enhanced AI integration, emphasizing Marimo's role in bridging the gap between data exploration and production-ready applications.
Announcements
Hello and welcome to the Data Engineering Podcast, the show about modern data management
Tired of data migrations that drag on for months or even years? What if I told you there's a way to cut that timeline by up to 6x while guaranteeing accuracy? Datafold's Migration Agent is the only AI-powered solution that doesn't just translate your code; it validates every single data point to ensure perfect parity between your old and new systems. Whether you're moving from Oracle to Snowflake, migrating stored procedures to dbt, or handling complex multi-system migrations, they deliver production-ready code with a guaranteed timeline and fixed price. Stop burning budget on endless consulting hours. Visit dataengineeringpodcast.com/datafold to book a demo and see how they're turning months-long migration nightmares into week-long success stories.
Your host is Tobias Macey and today I'm interviewing Akshay Agrawal about Marimo, a reusable and reproducible Python notebook environment
Interview
Introduction
How did you get involved in the area of data management?
Can you describe what Marimo is and the story behind it?
What are the core problems and use cases that you are focused on addressing with Marimo?What are you explicitly not trying to solve for with Marimo?
Programmatic notebooks have been around for decades now. Jupyter was largely responsible for making them popular outside of academia. How have the applications of notebooks changed in recent years?What are the limitations that have been most challenging to address in production contexts?
Jupyter has long had support for multi-language notebooks/notebook kernels. What is your opinion on the utility of that feature as a core concern of the notebook system?
Beyond notebooks, Streamlit and Hex have become quite popular for publishing the results of notebook-style analysis. How would you characterize the feature set of Marimo for those use cases?
For a typical data team that is working across data pipelines, business analytics, ML/AI engineering, etc. How do you see Marimo applied within and across those contexts?
One of the common difficulties with notebooks is that they are largely a single-player experience. They may connect into a shared compute cluster for scaling up execution (e.g. Ray, Dask, etc.). How does Marimo address the situation where a data platform team wants to offer notebooks as a service to reduce the friction to getting started with analyzing data in a warehouse/lakehouse context?
How are you seeing teams integrate Marimo with orchestrators (e.g. Dagster, Airflow, Prefect)?
What are some of the most interesting or complex engineering challenges that you have had to address while building and evolving Marimo?\
What are the most interesting, innovative, or unexpected ways that you have seen Marimo used?
What are the most interesting, unexpected, or challenging lessons that you have learned while working on Marimo?
When is Marimo the wrong choice?
What do you have planned for the future of Marimo?
Contact Info
LinkedIn
Parting Question
From your perspective, what is the biggest gap in the tooling or technology for data management today?
Closing Announcements
Thank you for listening! Don't forget to check out our other shows. Podcast.__init__ covers the Python language, its community, and the innovative ways it is being used. The AI Engineering Podcast is your guide to the fast-moving world of building AI systems.
Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
If you've learned something or tried out a project from the show then tell us about it! Email hosts@dataengineeringpodcast.com with your story.
Links
Marimo
Jupyter
IPython
StreamlitPodcast.__init__ Episode
Vector Embeddings
Dimensionality Reduction
Kaggle
Pytest
PEP 723 script dependency metadata
MatLab
Visicalc
Mathematica
RMarkdown
RShiny
Elixir Livebook
Databricks Notebooks
Papermill
Pluto - Julia Notebook
Hex
Directed Acyclic Graph (DAG)
Sumble Kaggle founder Anthony Goldblum's startup
Ray
Dask
Jupytext
nbdev
DuckDBPodcast Episode
Iceberg
Superset
jupyter-marimo-proxy
JupyterHub
Binder
Nix
AnyWidget
Jupyter Widgets
Matplotlib
Altair
Plotly
DataFusion
Polars
MotherDuck
The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA
Building the future of collaborative AI development with Akshay Agrawal
In this episode of Gradient Dissent, Akshay Agrawal, Co-Founder of Marimo, joins host Lukas Biewald to discuss the future of collaborative AI development.
They dive into how Marimo is enabling developers and researchers to collaborate seamlessly on AI projects, the challenges of scaling AI tools, and the importance of fostering open ecosystems for innovation. Akshay shares insights into building a platform that empowers teams to iterate faster and solve complex AI challenges together.
Follow Weights & Biases:
https://twitter.com/weights_biases
https://www.linkedin.com/company/wandb
Join the Weights & Biases Discord Server:
https://discord.gg/CkZKRNnaf3
marimo: Reactive Notebooks and Deployable Web Apps in Python
<p>What are common issues with using notebooks for Python development? How do you know the current state, share reproducible results, or create interactive applications? This week on the show, we speak with Akshay Agrawal about the open-source reactive marimo notebook for Python.</p>
<p>Before writing any code, Akshay wrote a 2,500-word design document. He wanted to create a maintainable and reproducible tool that avoided the hidden state of traditional notebooks. We discuss solving the hidden state problem by building the notebook as a directed acyclic graph (DAG).</p>
<p>Akshay shares how marimo notebooks are stored as pure Python files, which makes them easy to read, importable, and git-friendly. We discuss serializing package requirements using PEP 723 inline metadata to create standalone reproducible notebooks. We also cover how marimo notebooks can be deployed as a web app or dashboard using Pyodide.</p>
<div class="alert alert-primary" role="alert">
<p><strong>Course Spotlight:</strong> <a href="https://realpython.com/courses/navigate-namespaces-scope/">Navigating Namespaces and Scope in Python</a></p>
<p>In this course, you’ll learn about Python namespaces, the structures used to store and organize the symbolic names created during execution of a Python program. You’ll learn when namespaces are created, how they are implemented, and how they define variable scope.</p>
</div>
<p>Topics:</p>
<ul>
<li>00:00:00 – Introduction </li>
<li>00:02:06 – Akshay’s background and studies </li>
<li>00:04:14 – Work at Google and PhD program </li>
<li>00:06:29 – Sharing notebooks </li>
<li>00:08:18 – Starting work on marimo 2 years ago </li>
<li>00:12:48 – Avoiding notebook issues and building a DAG </li>
<li>00:18:39 – The difference of reactivity </li>
<li>00:20:39 – What is a marimo notebook? </li>
<li>00:23:39 – Video Course Spotlight </li>
<li>00:24:50 – Reproducibility and managing package requirements</li>
<li>00:27:49 – Using decorators for cells </li>
<li>00:30:23 – Writing a design document before any coding </li>
<li>00:34:08 – Interactivity and UI widgets </li>
<li>00:38:20 – Design decisions and built-in widgets </li>
<li>00:42:05 – Creating a deployable web application </li>
<li>00:44:34 – Exploring examples and tutorials </li>
<li>00:46:13 – Supporting DataFrame libraries with narwhals </li>
<li>00:48:00 – Migrating from a Jupyter notebook </li>
<li>00:52:02 – Working with cells and not running code </li>
<li>00:54:30 – A couple favorite tutorials </li>
<li>00:56:17 – What are you excited about in the world of Python? </li>
<li>00:57:39 – What do you want to learn next? </li>
<li>00:59:34 – How can people follow the project and yourself? </li>
<li>01:00:12 – Thanks and goodbye</li>
</ul>
<p>Show Links:</p>
<ul>
<li><a href="https://marimo.io/">marimo - a next-generation Python notebook</a></li>
<li><a href="https://www.youtube.com/watch?v=9R2cQygaoxQ">marimo: an open-source reactive notebook for Python - Akshay Agrawal (Nbpy2024) - YouTube</a></li>
<li><a href="https://www.tensorflow.org/">TensorFlow</a></li>
<li><a href="https://marimo.io/gallery">Made with marimo - marimo</a></li>
<li><a href="https://docs.marimo.io/faq.html#faq-problems">FAQ - marimo</a></li>
<li><a href="https://plutojl.org/">Pluto.jl — interactive Julia programming environment</a></li>
<li><a href="https://observablehq.com/">Observable: Build expressive charts and dashboards with code</a></li>
<li><a href="https://blog.jetbrains.com/datalore/2020/12/17/we-downloaded-10-000-000-jupyter-notebooks-from-github-this-is-what-we-learned/">We Downloaded 10,000,000 Jupyter Notebooks From Github – This Is What We Learned - The Datalore Blog</a></li>
<li><a href="https://leomurta.github.io/papers/pimentel2019a.pdf">A Large-scale Study about Quality and Reproducibility of Jupyter Notebooks</a></li>
<li><a href="https://marimo.io/blog/lessons-learned">Lessons learned reinventing the Python notebook - marimo</a></li>
<li><a href="https://realpython.com/podcasts/rpp/226/">Episode #226: PySheets: Spreadsheets in the Browser Using PyScript</a></li>
<li><a href="https://peps.python.org/pep-0723/">PEP 723 – Inline script metadata</a></li>
<li><a href="https://packaging.python.org/en/latest/specifications/inline-script-metadata/#inline-script-metadata">Inline script metadata - Python Packaging User Guide</a></li>
<li><a href="https://marimo.io/blog/sandboxed-notebooks">Serializing package requirements in marimo notebooks - marimo</a></li>
<li><a href="https://astral.sh/blog/uv-unified-python-packaging">uv: Unified Python packaging</a></li>
<li><a href="https://marimo.io/blog/newsletter-7">marimo Newsletter 7 - Jupyter to marimo</a></li>
<li><a href="https://docs.marimo.io/guides/integrating_with_marimo/custom_ui_plugins.html">Custom UI elements - marimo</a></li>
<li><a href="https://anywidget.dev/">anywidget - anywidget</a></li>
<li><a href="https://docs.marimo.io/guides/interactivity.html">Interactive elements - marimo</a></li>
<li><a href="https://realpython.com/podcasts/rpp/224/">Episode #224: Narwhals: Expanding DataFrame Compatibility Between Libraries</a></li>
<li><a href="https://calmcode.io/course/marimo/introduction">Calmcode - marimo: Introduction</a></li>
<li><a href="https://marimo.io/discord">Join the marimo Discord</a> </li>
<li><a href="https://marimo.io/newsletter">marimo newsletter</a> </li>
<li><a href="https://twitter.com/marimo_io">marimo on Twitter</a> </li>
<li><a href="https://www.linkedin.com/company/marimo-io/">marimo on LinkedIn</a> </li>
<li><a href="https://akshayagrawal.com/">Akshay Agrawal’s website</a> </li>
<li><a href="https://x.com/akshaykagrawal">Aksahy on Twitter</a></li>
</ul>
<p>Level up your Python skills with our expert-led courses:</p>
<ul>
<li><a href="https://realpython.com/courses/navigate-namespaces-scope/">Navigating Namespaces and Scope in Python</a></li>
<li><a href="https://realpython.com/courses/python-decorators-101/">Python Decorators 101</a></li>
<li><a href="https://realpython.com/courses/using-jupyter-notebooks/">Using Jupyter Notebooks</a></li>
</ul> <p><a rel="payment" href="https://realpython.com/join">Support the podcast & join our community of Pythonistas</a></p>
