- People of Research Newsletter
- Posts
- 💔 When Research Recommendations Break & 📈 Practical vs. Statistical Significance
💔 When Research Recommendations Break & 📈 Practical vs. Statistical Significance
📈 Practical vs. Statistical Significance, 💬 Why I Stopped Attending My UXR Interviews, 💔 When Research Recommendations Break - Join Research Lunch Club, now live in 33+ cities! Plus, our Member Interview from 🇺🇸 San Francisco, a podcast pick on Dan Ariely, author of Predictably Irrational, on unfakeable signals, friction, and trust in brands From Behavioral Science For Brands + the latests open roles in research!
People of Research
November 12, 2025
🎤 Research Community
🚀 Research Lunch Club November Matches Are Coming!
We’re connecting researchers across 20+ cities around the world — from New York to London to Sydney.
From AI to cognitive science, ethnography to UX research, members are being matched in trios to share ideas, stories, and insights over lunch. 🍽️
Whether you’re a seasoned researcher or just starting out, there’s a seat waiting for you.
🍴 Why not start this weekend with lunch alongside two brilliant researchers? It’s networking made simple — tailored, fun, and over food. Check out this weeks highlight from the around the globe 🌏️ 👇️
📰 Article Picks
🔗 Here in the Real World: The Importance of Practical Significance in A/B Testing: Alexander O. in "The Data Are Just Packed" argues that in A/B testing, "practical significance" is as crucial as statistical significance, urging practitioners to move beyond a sole focus on p-values. He defines practical significance as an effect large enough to matter in a real-world business context, using a hypothetical example where a customer lifetime value (CLV) increase is statistically significant but not large enough to outweigh the cost of the treatment. The author stresses that while statistical significance confirms that a result is likely not due to chance, practical significance answers the more important question: "Does this change actually matter enough for us to act on it?"
🔗 Why I Stopped Attending My Own User Research Interviews: Ethan Pierce explains his shift to asynchronous user research, arguing that an interviewer's presence inevitably introduces bias as participants try to "give me what they think I want." He found that by removing himself from the room and having users record their reactions to prompts, he received more candid and surprising feedback. Using his product, Adaptive Reader, as a case study, he describes how an unexpected, repeated question from asynchronous interviews ("Will there be paperbacks?") completely reshaped the product roadmap, leading to insights he might have otherwise missed. While acknowledging its limitations, he champions async research as a powerful complement for getting honest, unfiltered feedback, especially in early-stage discovery.
🔗 Insights Aren't Outcomes: Research Recommendation Breakage: Brian Utesch and Tammi Fitzwater for NN/g introduce the concept of "breakage," where valuable research recommendations are lost, diluted, or ignored before they can be implemented and reach the user. Using a retail analogy for damaged goods, they argue that breakage happens due to competing priorities, organizational silos, and fuzzy ownership, leading to wasted effort and researcher burnout. They stress that teams must move beyond simply delivering insights and hoping for action, advocating for a system to track recommendations through to adoption to ensure that the value chain from data to user outcome is not broken.
🔗 Everyone loves research — Until it's time to do some: Leo Bacevicius, in Bootcamp on Medium, explores the frustrating reality that while many teams claim to value user research, it often doesn't get done due to cultural barriers, lack of practice, or poor planning. He shares personal experiences of pushing for research in low-maturity organizations where the concept was foreign, and strategies for overcoming resistance, such as involving engineers in customer calls to build empathy and provide them with direct user context. Bacevicius emphasizes that conducting research is a skill that improves with practice and that the challenge is often as much about navigating internal stakeholder dynamics as it is about talking to users.
🎙️ Podcast Pick
💼 Job Picks
Are you a hiring manager looking to share research opportunities with a global network of over 5,000 researchers in more than 60 countries? We'd love to hear from you! Just drop us an email ✉️.
Europe & UK
North & South America
Asia & Oceania
👀 Sponsor us?
Our weekly newsletter is a fantastic way to connect with a global audience! With over 5,000 researchers tuning in from more than 60 countries, it's a great chance for sponsors to boost their brand visibility and make a real impact. - 💬 let’s talk!
👋 About us
We’re People of Research & Research Lunch Club - a global hybrid community uniting researchers across the industries, from UX Research to Behavioural Science. We connect, collab and grow. People of Research was created by @faysel.
Reply