November 29, 2024
I hope this message finds you well. It’s the CRO Roundtable Roundup!
Thanks to Matt Beischel, Iqbal Ali, Craig Sullivan, and Veronika Morozová for joining us. Want to get in on the action and talk with cool CRO people like this? Then join us this Friday, December 6.
One Sentence Takeaway
This Time We Discussed:
- Issues with A/A testing
- What happens if an A/A reaches statistical significance?
- Transitioning to server-side testing
- Increased technical complexity tradeoffs
- Are the results more trustworthy than client-side results?
- Getting Quality Assurance (QA) Done
- Detecting test errors early
- Using Miro boards for collaborative QA
- Gamifying repetitive QA tasks to engage team members
- Data Visualization Techniques
- The benefits of icon arrays (e.g., 4 out of 5) versus percentages (80%)
- Representing uncertainty visually in A/B test results
- The Whos and Hows of Stakeholder Communication
- Tailoring reports and presentations for technical versus executive stakeholders
- The difference between reporting and analysis
- Waiting for tests to “cook” before interpreting results
- Using your Creativity in Experimentation
- Using A/B testing principles in personal projects
- Applying storytelling frameworks to experiment reports
Quotes of the Week
Standout quips from this week’s roundtable
- “Nobody talks about Fight Club until the sample is in.” – Craig Sullivan
- “Effective data visualization isn’t about the data you want to include but the action it inspires.” – Matt Beischel
CRO Link Digest
Links to interesting, amusing, thought-provoking, or downright silly content shared at this week’s roundtable
- No links this week, but lots of books instead!
CRO Book Club
by Andy Kirk
This handbook offers everything students and scholars need to master the craft of developing insightful and delightful data visualisations. Across over 300 pages packed full of useful knowledge this book is an essential reference to help readers harness the wide range of contextual, analytical, editorial, and visual ingredients that shape this complex but invigorating subject. With an emphasis on critical thinking over technical instruction, the importance of good decision-making is placed at the centre of a proven step-by-step process. Blending conceptual, theoretical, and practical thinking, this updated edition will inspire you to elevate your ambition and inform you how to get there. – Amazon Link (non-referral)
by Annie Cushing
There are a lot of Excel books on the market. So why write another one? First of all, most Excel books are published in black and white. So they can’t focus much (if at all sometimes) on the art of data visualization because you can’t say, “Imagine that lighter gray is orange, the dark gray is blue, and the murky gray is…well, that’s actually gray. Making Data Sexy is a comprehensive, full-color book with more than 500 screenshots and diagrams and covers 60+ visualizations. It has been created for people who need to learn Excel but don’t like to actually read. – Amazon Link (non-referral)
by Edward R. Tufte
The classic book on statistical graphics, charts, tables. Theory and practice in the design of data graphics, 250 illustrations of the best (and a few of the worst) statistical graphics, with detailed analysis of how to display data for precise, effective, quick analysis. Design of the high-resolution displays, small multiples. Editing and improving graphics. The data-ink ratio. Time-series, relational graphics, data maps, multivariate designs. Detection of graphical deception: design variation vs. data variation. Sources of deception. Aesthetics and data graphical displays. – Amazon Link (non-referral)
by David Bradford and Carole Robin
Connect shows why relationship-building is crucial to becoming a more effective manager and leader and living a fulfilled life, from highlighting the importance of curiosity and empathy to demonstrating how to break logjams and negotiate boundaries. Filled with research-backed insights, useful concepts and thought-provoking exercises, Connect is an important resource for anyone hoping to build and sustain relationships, providing tools to make relationships robust — and even exceptional. – Amazon Link (non-referral)
by Dawna Markova and Angie McArthur
Collaborative intelligence, or CQ, is a measure of our ability to think with others on behalf of what matters to us all. It is emerging as a new professional currency at a time when the way we think, interact, and innovate is shifting. In the past, “market share” companies ruled by hierarchy and topdown leadership. Today, the new market leaders are “mind share” companies, where influence is more important than power, and success relies on collaboration and the ability to inspire. – Amazon Link (non-referral)
Off-Topic Sidebars
Experimentation isn’t the only thing we talk about at the CRO Roundtable. There’s often a healthy dose of discussion on shared interests, personal passions, and hobbies. This week we had some sidebars about:
- Thanksgiving traditions and recipes
- Global food nutrient decline
- AI-enhanced cocktail recipe generation
- Using Gmail “plus” tricks for multiple free trials