Why we need to get rid of significance in A/B testing

P-values and significance are utterly ill-suited tools for A/B testing.

Maximilian Speicher
UX Collective
Published in
6 min readMar 11, 2022

--

A door with two mailboxes. Mailbox “A” on the left, mailbox “B” on the right.
Photo by Jason Dent on Unsplash

In my previous job, I was responsible for all A/B testing in an e-commerce department. We worked with an external partner and took care of the whole package — from generating hypotheses to developing the tests to providing a report of the results. Those…

--

--

A designer who writes about leadership, strategy, & anything UX • Doctor of Computer Science • formerly University of Michigan • maxspeicher.com/newsletter