Road construction next 2,790 miles. It’s the CRO Roundtable Roundup!
Thanks to Matt Beischel, Iqbal Ali, Craig Sullivan, and Jorden Lentze for joining us. Want to get in on the action and talk with cool CRO people like this? Then join us this Friday, March 7.
One Sentence Takeaway
I’m a busy person; give me the TL:DR
Prioritizing QA in experimentation is essential to prevent costly mistakes, ensure accuracy, and streamline processes.
The Notes
What we discussed this week
- QA Processes for Experiments
- Importance of QA in preventing costly mistakes
- Balancing efficiency and thoroughness when getting sign-offs
- Process automation at scale for experiment validation
- Doing QA on statistical planning and design stages
- Formalized QA sessions involving all team members
- Booking.com’s approach to QA and large-scale testing
- Automated systems for stopping faulty experiments
- AI’s Impact on Jobs and Experimentation
- Replacing low and high-level development roles
- Leadership’s misunderstanding of AI capabilities and cost savings
- Companies eliminating entire departments due to AI-driven efficiencies
- AI assisting with A/B test implementation and automation
- Long-term impact on productivity and job displacement
- Experimentation at Scale
- Microsoft’s claim of running 100K tests a year
- The challenge of defining and counting experiments
The Quotes
Standout quips from this week
- “You’ve got to pump those numbers up, kid.” — Matt Beischel
- “If you don’t know your own product, how can you actually be a product manager?” — Jorden Lentze
- “It’s not bad enough for the masses to revolt yet.” — Matt Beischel
Book Club
Relevant reads recommended this week
It’s easy to look at the state of the world around us and feel hopeless. We live in an era marked by war, climate crisis, political polarization, and acute inequality—and yet many of us feel powerless to do anything about these profound issues. We’ve been assured that unfettered capitalism is necessary to ensure our freedom and prosperity but why, in our age of unchecked corporate power, are most of us living paycheck to paycheck? When the economy falters, why do governments bail out corporations and shareholders but leave everyday people in the dust?
Buy This Book ←non-referral link
CRO Link Digest
Useful and thought-provoking content shared this week
- AI Product Management – blog article for Silicon Valley Product Group by Marty Cagan and Marily Nika
- The End of Programming as We Know It – blog article for O’Reilly by Tim O’Reilly
Off-Topic Sidebars
Experimentation isn’t the only thing we talk about at the CRO Roundtable. There’s often a healthy dose of discussion on shared interests, personal passions, and hobbies.
- James Bond villains are looking kind of realistic
- Moonraker’s villain parallels a certain prominent billionaire
- Driving fast on the M25 in a Lotus Esprit
- Off-the-beaten-path spots to visit in London
- Problems in modern economics
- Housing affordability and generational wealth gaps
- Universal Basic Income (UBI) as a potential solution
- Capitalism’s structure favoring the upper class
- Historical parallels to the Luddite movement
- Corporate tax loopholes and offshore money movements
Sidebar Shareables
Amusing sidebar content shared this week
- Secret London – ultra-shareable online guide to news, events, and things to do in London
- Barts Pathology Museum – medical museum housing one of the largest collections of human pathological specimens in the UK
- Can’t Get You Out of My Head – part one of a six-part documentary series investigating the history of modern political populism by Adam Curtis