SmartBear and Multi-Agent QA
AI coding tools have dramatically accelerated the pace of development, and the bottleneck in the software development lifecycle has shifted to code validation and testing. However, the conventional tools and workflows that QA teams have relied on were not designed for a world where a single engineer can generate thousands of lines of code in a day.
SmartBear is a software quality platform spanning test automation, API lifecycle management, and observability. The company recently launched an AI-native QA platform called BearQ, which deploys autonomous agents that explore web applications, learns their structure and behavior, and authors and maintains test cases continuously.
Fitz Nowlan is the VP of AI and Architecture at SmartBear and the co-founder of Reflect, which is a web testing platform acquired by SmartBear in 2024. In this episode, Fitz joins Kevin Ball to discuss why web UI testing is uniquely challenging, how BearQ’s multi-agent architecture coordinates exploration and testing, why test data management becomes a hard distributed systems problem at scale, and what agentic development means for the future of QA.
Full Disclosure: This episode is sponsored by SmartBear.
Kevin Ball or KBall, is the vice president of engineering at Mento and an independent coach for engineers and engineering leaders. He co-founded and served as CTO for two companies, founded the San Diego JavaScript meetup, and organizes the AI inaction discussion group through Latent Space.
Please click here to see the transcript of this episode.
Sponsorship inquiries: sponsor@softwareengineeringdaily.com
The post SmartBear and Multi-Agent QA appeared first on Software Engineering Daily.
#332 Dan Faulkner: The Code Is Clean. The App Is Broken. Why AI Development Has an Integrity Problem
What happens when AI writes code faster than anyone can test it?
In this episode of Eye on AI, Craig Smith sits down with Dan Faulkner, CEO of SmartBear, to explore one of the most underappreciated risks of the AI coding boom. As tools like Claude Code and Codex push software development to unprecedented speed, the systems built to validate that software are being left behind. Dan makes a distinction that every engineering leader needs to hear: clean code passing unit tests is not the same as an application that actually works.
Dan introduces the concept of application integrity, continuous and measurable assurance that your software does everything it was intended to do and nothing it was not. He explains why the gap between what AI builds and what teams actually validate is already creating hidden risk in production, and why that risk compounds the faster you ship.
We also get into the new failure modes that agentic AI is introducing. Slop squatting, instruction inversion, cascading errors. These are not theoretical. They are happening now, at scale, in codebases that no human has fully read.
Dan also walks through SmartBear's autonomy ladder framework and their newest product BearQ, a team of AI agents that explores your application, builds a knowledge graph, authors tests, runs them, and updates everything as your app evolves. The key distinction: it is built to augment human teams, not replace them.
Finally, Dan shares his honest take on the future of software engineering. The fallacy was always that coding was the hard part. The hard part is knowing what to build. That skill is not going anywhere.
Subscribe for more conversations with the people shaping the future of AI and emerging technology.
Stay Updated:
Craig Smith on X: https://x.com/craigss
Eye on A.I. on X: https://x.com/EyeOn_AI
(00:00) Introduction and Dan Faulkner's Background
(01:05) What SmartBear Does: Testing and API Lifecycle Management
(03:27) AI Is Outpacing Application Testing
(07:51) Slop Squatting, Instruction Inversion and New AI Failure Modes
(17:31) Black Boxes, Technical Debt and the Expertise Crisis
(22:00) How to Avoid Self-Validating AI Systems
(24:11) The Autonomy Ladder and BearQ
(31:30) Why Testing Must Be Continuous and Everywhere
(36:31) Infrastructure Risk and Automation Bias
(44:11) The Future of QA and New Specialist Roles
(50:44) How Teams Use SmartBear Tools Today
(58:57) The Future of Software Engineering and Human Roles
How can you test your code when you don’t know what’s in it?
Ryan hosts SmartBear’s VP of AI and Architecture Fitz Nowlan to explore how we’re moving away from old assumptions about software development, the challenges of testing MCP servers as LLM-driven agents introduce non-determinism that breaks tradition, and how data locality and data construction are becoming more valuable when source code is so easy to generate.
Episode notes:
SmartBear gives devs tools for application performance monitoring, software development, software testing, and API management—all at AI speed and scale.
Connect with Fitz on LinkedIn and email him at FitzNowlan@SmartBear.com
Congrats to Great Answer winner Alexander for winning the badge for their answer to Is there a way to make Runnable's run() throw an exception?.
TRANSCRIPT
See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.
Episode 817 | Bootstrapping in the Age of AI with Jason Cohen
How would a 2x unicorn founder build his next startup with AI?
In this episode, Rob Walling sits down with Jason Cohen, founder of SmartBear and WP Engine, to talk about building billion-dollar businesses, the future of AI for founders, and what makes small companies thrive even when the odds are stacked against them.
They dig into the early days of WP Engine, how Jason develops his frameworks, why execution beats ideas, and Jason’s framework for identifying “hidden multipliers” small, systematic changes that make an outsized impact.
Episode Sponsor:
Hiring engineers shouldn’t feel like sorting through AI-polished resumes.
G2i cuts through all of that. They’ve pre-vetted over 8,000 engineers, all with 5+ years of real experience, and they run live, human-led technical interviews to verify actual skills.
No time wasters. No guesswork. Just solid developers who can deliver.
G2i is trusted by companies like Meta, Microsoft, and countless bootstrapped founders who need to move fast without making expensive mistakes.
Get a 7-day free trial and $1,500 off when you mention Startups for the Rest of Us at https://www.g2i.co/rob
Topics we cover:
(03:45) – The core idea behind Hidden Multipliers
(09:24) – Writing as a way of thinking
(12:34) – Why sharing your frameworks matters
(14:14) – The origin of “Designing the Ideal Bootstrap Business”
(18:10) – The hidden weak links in every startup
(21:25) – De-risking and niching down effectively
(24:56) – Why narrowing your focus expands your reach
(26:24) – Building WP Engine in a commodity market
(29:37) – Out-executing funded competitors
(31:52) – Finding product–market resonance through pricing
(32:40) – How brand actually develops
(37:54) – Building in the age of AI: pitfalls and opportunities
(41:52) – The three categories of AI startups today
(46:02) – Why 10x improvement is the new baseline for differentiation
(49:19) – The real moat in the age of AI
Links from the Show:
MicroConf US 2026 – Portland, April 14–16, 2026 Promo Code: Rob50 for $50 off
The SaaS Playbook
PREORDER Hidden Multipliers by Jason Cohen
Designing the Ideal Bootstrapped Business with Jason Cohen
A Smart Bear Blog
Jason Cohen (@asmartbear) | X
If you have questions about starting or scaling a software business that you’d like for us to cover, please submit your question for an upcoming episode. We’d love to hear from you!
Subscribe & Review: iTunes | Spotify
5 questions to ask when your product stops growing | Jason Cohen (2x unicorn founder)
Jason Cohen is a four-time founder (including two unicorns, one being WP Engine) and an investor in over 60 startups, and has been sharing his lessons on company building at A Smart Bear for nearly 20 years. In this episode, Jason shares his methodical five-step framework for diagnosing stalled growth—a problem that faces almost every team.
We discuss:
1. Jason’s five-step framework: logo retention, pricing, NRR, marketing channels, target market
2. A small tweak that’ll double response rates on your cancellation surveys
3. Why “it’s too expensive” is almost never the real reason customers cancel
4. The “elephant curve” of growth
5. How repositioning the same product can increase revenue 8x
6. When to reconsider if growth is even the right goal for your business
—
Brought to you by:
10Web—Vibe coding platform as an API
Strella—The AI-powered customer research platform
Brex—The banking solution for startups
—
Episode transcript: https://www.lennysnewsletter.com/p/why-your-product-stopped-growing
—
Archive of all Lenny's Podcast transcripts: https://www.dropbox.com/scl/fo/yxi4s2w998p1gvtpu4193/AMdNPR8AOw0lMklwtnC0TrQ?rlkey=j06x0nipoti519e0xgm23zsn9&st=ahz0fj11&dl=0
—
Where to find Jason Cohen:
• Preorder Jason’s book: https://preorder.hiddenmultipliers.com/
• X: https://x.com/asmartbear
• LinkedIn: https://www.linkedin.com/in/jasoncohen
• Blog: https://longform.asmartbear.com
• Website: https://wpengine.com
—
Where to find Lenny:
• Newsletter: https://www.lennysnewsletter.com
• X: https://twitter.com/lennysan
• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/
—
In this episode, we cover:
(00:00) Introduction to Jason Cohen
(05:19) Jason’s writing journey
(08:25) Questions to ask when your product stops growing
(18:17) Getting real customer feedback
(20:27) Analyzing cancellation reasons
(26:54) Onboarding and activation
(29:35) Quick summary
(35:46) Revisiting pricing strategies
(41:46) Positioning strategies
(47:52) Why pricing is inseparable from your strategy
(52:06) The importance of net revenue retention (NRR)
(01:00:25) Asking whether or not this is good for the customer
(01:04:34) Leveraging existing customers
(01:06:42) Are your acquisition channels saturated? The “elephant curve”
(1:09:41) Why all marketing channels eventually decline
(01:12:04) Direct vs. indirect marketing channels
(1:13:36) Getting creative with new channels
(01:19:04) Do you actually need to grow?
(01:25:57) Deciding when to quit
(01:29:27) Book announcement
(01:33:21) AI corner
(01:34:35) Contrarian corner
(01:37:43) Lightning round and final thoughts
—
Referenced:
• Tyler Cowen’s website: https://tylercowen.com
• How to Perform a Customer Churn Analysis (and Why You Should): https://www.groovehq.com/blog/learn-from-customer-churn
• Linear: https://linear.app
• Jira: https://www.atlassian.com/software/jira
• Patrick Campbell’s post on X about pricing: https://x.com/Patticus/status/1702313260547006942
• The art and science of pricing | Madhavan Ramanujam (Monetizing Innovation, Simon-Kucher): https://www.lennysnewsletter.com/p/the-art-and-science-of-pricing-madhavan
• Pricing your AI product: Lessons from 400+ companies and 50 unicorns | Madhavan Ramanujam: https://www.lennysnewsletter.com/p/pricing-and-scaling-your-ai-product-madhavan-ramanujam
• Pricing your SaaS product: https://www.lennysnewsletter.com/p/saas-pricing-strategy
• M&A, competition, pricing, and investing | Julia Schottenstein (dbt Labs): https://www.lennysnewsletter.com/p/m-and-a-competition-pricing-and-investing
• “Sell the alpha, not the feature”: The enterprise sales playbook for $1M to $10M ARR | Jen Abel: https://www.lennysnewsletter.com/p/the-enterprise-sales-playbook-1m-to-10m-arr
• Buffer: https://buffer.com
• AG1: https://drinkag1.com
• How to find hidden growth opportunities in your product | Albert Cheng (Duolingo, Grammarly, Chess.com): https://www.lennysnewsletter.com/p/how-to-find-hidden-growth-opportunities-albert-cheng
• How Duolingo reignited user growth: https://www.lennysnewsletter.com/p/how-duolingo-reignited-user-growth
• The Elephant in the room: The myth of exponential hypergrowth: https://longform.asmartbear.com/exponential-growth
• HubSpot: https://www.hubspot.com
• Zigging vs. zagging: How HubSpot built a $30B company | Dharmesh Shah (co-founder/CTO): https://www.lennysnewsletter.com/p/lessons-from-30-years-of-building
• Adjacency Matrix: How to expand after PMF: https://longform.asmartbear.com/adjacency/
• Ecosystem is the next big growth channel: https://www.lennysnewsletter.com/p/ecosystem-is-the-next-big-growth
• ChatGPT apps are about to be the next big distribution channel: Here’s how to build one: https://www.lennysnewsletter.com/p/chatgpt-apps-are-about-to-be-the
• 10 contrarian leadership truths every leader needs to hear | Matt MacInnis (Rippling): https://www.lennysnewsletter.com/p/10-contrarian-leadership-truths
• Breaking the rules of growth: Why Shopify bans KPIs, optimizes for churn, prioritizes intuition, and builds toward a 100-year vision | Archie Abrams (VP Product, Head of Growth at Shopify): https://www.lennysnewsletter.com/p/shopifys-growth-archie-abrams
• Geoffrey Moore on finding your beachhead, crossing the chasm, and dominating a market: https://www.lennysnewsletter.com/p/geoffrey-moore-on-finding-your-beachhead
• ER on Prime Video: https://www.amazon.com/ER-Season-1/dp/B0FWK5WJQ4
• The Pitt on Prime Video: https://www.amazon.com/The-Pitt-Season-1/dp/B0DNRR8QWD
• Wispr Flow: https://wisprflow.ai
• Anker: https://www.anker.com
—
Recommended books:
• Will: https://www.amazon.com/Will-Smith/dp/1984877925
• Monetizing Innovation: How Smart Companies Design the Product Around the Price: https://www.amazon.com/Monetizing-Innovation-Companies-Design-Product/dp/1119240867
• Hidden Multipliers: Small Things That Accelerate Growth: https://preorder.hiddenmultipliers.com
• On Writing Well: The Essential Guide to Mastering Nonfiction Writing and Effective Communication: https://www.amazon.com/Writing-Well-Classic-Guide-Nonfiction/dp/0060891548
• Crossing the Chasm, 3rd Edition: The Updated Version of the Insightful Guide on Bringing Cutting-Edge Products to the Mainstream: https://www.amazon.com/Crossing-Chasm-3rd-Disruptive-Mainstream/dp/0062292986
—
Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.
—
Lenny may be an investor in the companies discussed.
To hear more, visit www.lennysnewsletter.com
This AI Startup Idea Has 500 Million Customers Waiting For It
Episode 554: Shaan Puri (https://twitter.com/ShaanVP) and Sam Parr (https://twitter.com/theSamParr) brainstorming unicorn startup ideas with $1B founder Jason Cohen.
Want to see Sam and Shaan’s smiling faces? Head to the MFM YouTube Channel and subscribe - http://tinyurl.com/5n7ftsy5
—
Show Notes:
(0:00) Intro
(3:00) AI improved presentations
(10:30) Automated Twitter hacking
(13:00) Being a billionaire tinkerer
(17:30) Rich vs. King
(22:30) All startups are screwed up--including the ones that work
(30:30) Be so good you can't fail
(32:30) Consumers don't value their time
(35:30) Will WP Engine go public?
(37:00) Lessons from Silver Lake Capital elites
—
Links:
• Smart Bear - https://smartbear.com/
• Jason Cohen on Twitter - https://twitter.com/asmartbear
• Zeck - https://www.zeck.app/
• Rich v. King - https://longform.asmartbear.com/rich-vs-king-sold-company
• Box.com - http://box.com/
• Silver Lake - https://www.silverlake.com
—
Check Out Sam's Stuff:
• Hampton - https://www.joinhampton.com/
• Ideation Bootcamp - https://www.ideationbootcamp.co/
• Copy That - https://copythat.com
• Hampton Wealth Survey - https://joinhampton.com/wealth
—
Check Out Shaan's Stuff:
Need to hire? You should use the same service Shaan uses to hire developers, designers, & Virtual Assistants → it’s called Shepherd (tell ‘em Shaan sent you): https://bit.ly/SupportShepherd
Past guests on My First Million include Rob Dyrdek, Hasan Minhaj, Balaji Srinivasan, Jake Paul, Dr. Andrew Huberman, Gary Vee, Lance Armstrong, Sophia Amoruso, Ariel Helwani, Ramit Sethi, Stanley Druckenmiller, Peter Diamandis, Dharmesh Shah, Brian Halligan, Marc Lore, Jason Calacanis, Andrew Wilkinson, Julian Shapiro, Kat Cole, Codie Sanchez, Nader Al-Naji, Steph Smith, Trung Phan, Nick Huber, Anthony Pompliano, Ben Askren, Ramon Van Meer, Brianne Kimmel, Andrew Gazdecki, Scott Belsky, Moiz Ali, Dan Held, Elaine Zelby, Michael Saylor, Ryan Begelman, Jack Butcher, Reed Duchscher, Tai Lopez, Harley Finkelstein, Alexa von Tobel, Noah Kagan, Nick Bare, Greg Isenberg, James Altucher, Randy Hetrick and more.
—
Other episodes you might enjoy:
• #224 Rob Dyrdek - How Tracking Every Second of His Life Took Rob Drydek from 0 to $405M in Exits
• #209 Gary Vaynerchuk - Why NFTS Are the Future
• #178 Balaji Srinivasan - Balaji on How to Fix the Media, Cloud Cities & Crypto
• #169 - How One Man Started 5, Billion Dollar Companies, Dan Gilbert's Empire, & Talking With Warren Buffett
• #218 - Why You Should Take a Think Week Like Bill Gates
• Dave Portnoy vs The World, Extreme Body Monitoring, The Future of Apparel Retail, "How Much is Anthony Pompliano Worth?", and More
• How Mr Beast Got 100M Views in Less Than 4 Days, The $25M Chrome Extension, and More
How Scaling Turns Rare Occurrences Into Common Ones with Jason Cohen
Today Corey Quinn is joined by Founder and Chief Innovation Officer at WP Engine, Jason Cohen. Jason breaks down the biggest issues he has seen throughout his career hosting millions of websites including why seemingly rare problems should be expected at scale, how moving on after attaining a “good enough” metric can save time and money, and what it means to be proud of your work in the world of cybersecurity. Check it out!
Show Highlights
(00:00) - WordPress popularity and outsourcing engineering tasks
(07:28) - Web hosting and scalability
(11:01) - Server reliability and quality control
(14:18) - Scaling infrastructure and prioritizing customer value
(26:20) - Website speed and optimization
(28:17) - WordPress scalability and deployment in a cloud environment
(36:14) - Customer profitability and service limitations
(38:54) - Security measures for ethical decision-making
(47:19) - Balancing free speech and decision-making in online content moderation
About Jason
Founder of unicorn WP Engine (200,000 customers, 1,200 employees). Previously founder of bootstrapped Smart Bear (sold 2008; re-sold in 2021 at ~$2B) and ITWatchDogs (sold 2004). Original mentor and angel investor with Austin-based Capital Factory since 2009.
Written about startups for seventeen years, most recently at https://longform.asmartbear.com; Twitter: @asmartbear.
Links Referenced:
Personal Website: https://longform.asmartbear.com/
WP Engine: https://wpengine.com/
Linkedin: https://www.linkedin.com/in/jasoncohen/
Visible APIs get reused, not reinvented
With so many companies offering API products, it can be hard to get your particular APIs discovered and used by the developers who need them most. You might have the best, most useful solutions out there, but if you’re relying on the digital equivalent of foot traffic for discoverability, it might as well not exist. And if an API solution can’t be found, then someone else is going to reinvent it.
On this sponsored episode, we chat with SmartBear API Technical Evangelist Frank Kilcommins about the growing challenges of API visibility and how to outsmart the invisibility trap with the right development strategies and tools.
Episode notes:
Kilcommins suggests you can get better visibility for your APIs with SmartBear's new free API exploration tool.
Open specifications like the Open API Initiative help make your endpoints easier to understand—both by humans and computers.
Connect with Frank Kilcommins on Twitter and LinkedIn.
Congrats to Stack Overflow user WorstCase, who asked five well-received questions on five separate days and earned themselves a shiny new Curious badge.
See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.
SaaStr 613: How Top CMOs Drive Growth in the Age of the Self-Serve Buyer with Klaviyo SVP of Marketing Kady Srinivasan, Talkdesk CMO Kathie Johnson, SmartBear CMO Cynthia Gumbert and TrustRadius Founder and CEO Vinay Bhagat
To scale your business customers need to be at the center of all you do. The reason for your business is to solve their largest problems and be mission-critical to their success. If your customers succeed your business will too. However, what if you can't find or engage your target buyers? The way technology is bought and sold today has dramatically changed. The buyer's journey is not linear but something they want to control. According to Bain and Co study, 94% of prospective buyers are well informed about a product before ever engaging with a sales team. From our research at TrustRadius, virtually 100% of buyers want to self-serve, 58% of buyers say word-of-mouth is a top source for discovery and evaluation. We'll evaluate these buyer trends and provide strategies to find and win these new-age customers now and in the future.
Full video: https://youtu.be/eGL9ndS3Kmk
Want to join the SaaStr community? We're the 🌎largest community for B2B software.
Subscribe for weekly updates: https://www.saastr.com/subscribeform
Twitter: https://twitter.com/saastr
LinkedIn: https://www.linkedin.com/company/2724976
Quora Group: https://www.quora.com/q/cloud
Facebook: https://www.facebook.com/SaaStr/
Instagram: https://www.instagram.com/saastr/
Our North American Event: https://bit.ly/2OXeAYh
Our European Event: https://bit.ly/2OZTad8
#088 – Sage Advice (and Reasons You Probably Won't Follow It) from Jason Cohen of WP Engine
Not only has Jason Cohen (@asmartbear) bootstrapped a software company from $0 to over $1M in revenue, but he's done it four times! The stories behind Jason's successes are plastered all over the Internet for anyone to find, so I decided to take a different approach: I skipped Jason's backstory and instead proceeded to squeeze him like a sponge to extract every ounce of advice I possibly could in the hour we had together. The result is a wide-ranging discussion about the best path for reaching your first $10k/month in revenue, the lies we tell ourselves as founders, and why you probably won't take the advice that Jason (or anyone else for that matter) gives you.
Transcript, speaker information, and more: https://www.indiehackers.com/podcast/088-jason-cohen-of-wp-engine
Ep. 8: Better Beer Through AI
Whether brewing hearty stouts or crisp lagers, flavor is a fickle thing. Not only is it hard to create consistently good brew, as humans our ability to identify - and remember - flavors is flawed. Yet brands worth billions rely on creating consistent flavors. We talk to Jason Cohen, founder of Gastrograph, who is using AI to help businesses that create beer, chocolate, wine, coffee, and spirits better understand flavor.