September 4, 2024
It’s a state of mind. It’s the CRO Roundtable Roundup!
Thanks to Matt Beischel, Iqbal Ali, and Craig Sullivan for joining us. Want to get in on the action and talk with cool CRO people like this? Then join us this Friday, October 11.
One Sentence Takeaway
AI integration in CRO is being pursued but lacks clarity, leading to challenges with existing scoring models and subjective methodologies.
This Time We Discussed:
-
AI and Experimentation in CRO
- Applying AI to prioritization models such as ICE scoring lacks substance and practical applicability
-
The Quality of Quality Assurance (QA) in Various Testing Tools
- Convert’s limitations in handling IP-specific tests and cookie scope
- VWO has decent QA features, but still has issues
- Sitespect and Maxymizer have robust QA suites
- Webtrends Optimize offers multi-test visibility
- Building Custom QA Tools
- Custom plugins lack cross-platform compatibility
- Potential collaboration to create a QA alliance or shared plugin among vendors
- Difficulty in maintaining query strings for live tests and interactions
- Using cookie-scoping instead of URL-scoping for more stability
-
Agency Partnerships
- Partnership agreements often offer discounts but lack mutual benefits for agencies
- Range of discounts varies widely, impacting long-term partnerships
- Discounts are a minor aspect; relationships and collaboration are more critical
-
Challenges with Language Models
- AI models trained on specific English styles, unfairly flagging African and Indian writers
- Language biases in AI tools can marginalize non-Western English speakers
Quotes of the Week
Standout quips from this week’s roundtable
- “So, like, what’s AI even gonna do on a subjective scoring method?” – Matt Beischel
- “If you fed it garbage, you’ll still get garbage, but faster.” – Craig Sullivan
CRO Link Digest
Links to interesting, amusing, thought-provoking, or downright silly content shared at this week’s roundtable
- Cursor: An IDE with AI built-in – Freeware Software Vendor
- Gabriel Udoh: AI Ethics Researcher – LinkedIn Profile Page
- Can AI Beat Humans at Creativity? – LinkedIn Post by Craig Sullivan
Off-Topic Sidebars
Experimentation isn’t the only thing we talk about at the CRO Roundtable. There’s often a healthy dose of discussion on shared interests, personal passions, and hobbies. This week we had some sidebars about:
- Favorite travel destinations and flight experiences
- What to do with your pets when you travel
- Physical product design flaws you’ve experienced firsthand
Off-Topic Shareables
- No off-topic shareables this week, sorry!