Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

What fantasy world are you living in where you think Netflix and Facebook don't already do this? They constantly run paid interviews with people to source this information.

The issue is many-fold: 1) formalising multiple half hour interviews into something you can A/B test is non-trivial, 2) given the logistics of these interviews the sample you end up with is small and therefore bias.



I worked for the company running the most A/B tests in the industry for most of my career.

Having a user testing session is relatively rare, saved for special one-off products/features. Actually having feedback from those sessions distilled into product feature proposals, and getting them into the product roadmap is even rarer.

In the meantime, hundreds of A/B tests based on metrics like these would have been ran and resulted in a permanent change to the product. It’s just so much easier for everyone involved to trust “the data” vs something that requires nuanced interpretation.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: