I don’t believe in data-driven design
There, I said it.
We like to think — we want to think — everything has a meaning, a purpose. We want to trust a company when it says it is not doing what we’ve heard or read it is doing that we believe is ”bad” or ”unethical”. We want to believe people are not inherently evil or mischievous, that they behave erratically only from time to time and not truly on purpose.
The same holds for design.
We build principles, establish rules to follow — or to break. We devise methods and develop systems. We use design thinking and design sprints. We conduct in-depth interviews, build prototypes, measure conversion rates, track eye movement, and non-verbal responses. We test social engagement and sample our reach in target demographics.
It seems we have a Voigt-Kampff test already in place to help us sell some bullshit products, but that’s a whole other story for some other time.
All of this is our way to test a very straightforward hypothesis:
Are we right, or are we wrong?
In marketing lingo: will it sell or not?
We gather the data, and we analyze it. We build personas to have an avatar, an approximation of our „average user.” We present our findings to stakeholders, shareholders, and anyone in charge of deciding on the direction we should follow. We then lunge forward and design the shit out of our products because the data said so.
And then suddenly it doesn’t work all that well, and the difference in sales, conversion, or any other magic KPI we established as a definition of success — makes not that much of a difference.
Then we try to find the answers „why it’s not working?”. We look and scan the data once more. We browse through our findings. We go back to drawing boards, to our flows, wireframes, and diagrams. We look for solutions, sometimes we blame someone or try to find a scapegoat in factors outside of our domain.
But I think we seem to be forgetting one simple fact — at the end of the line, it’s people we are dealing with here and…
People are not data.
The data is true.
The numbers don’t lie. Ones and zeros have no plan, no agenda, no left or right views, no cause to fight for, no mission or objective, no guides or rules to follow, no promise to keep or break, no one to support or disappoint. Data has no morals, no ethics, no systems of beliefs. Data is perfect.
People, on the other hand…
Well, we are far from being perfect simply because we are people.
We make mistakes. We forget. We cheat, and we lie — sometimes with intent, sometimes not even registering the act. We want what’s best for „us” and we (mostly) want it now. We tend to not understand other cultures, foreign ideas, or different points of view. We speak and understand mostly one language (if that). We have our agendas, causes we fight for, missions, and objectives. We have our morals, ethics, and sets of values, sometimes truly different from one another.
And we are soooooo easy to manipulate.
Like-farming, fake influencers, dark patterns, content farms. The list goes on. And we all believe what we’ve read or seen on the internets. Because, according to the classic meme, why would anybody go on the internet and lie, right? We stick to our thought silos, we rarely venture out of our comfort zones, we tend to have a rather narrow view of reality. Our behavior is skewed from the start then, and all data created from this point on must be — by definition — limited.
But, surprisingly, it’s the same data designers use to base their decisions on and design their products.
And since we base our decisions on the data generated by people, sometimes using manipulation mechanics, it’s easy to overestimate that data’s value.
And my point, in a nutshell, is
data has become kinda useless (in itself at least).
I’ve always been skeptical about statistics in general, especially the idea of a median, but when you start to consider the data in the aspect of social media, it makes little to no sense. If you can manipulate something so ubiquitous as a Facebook post, what’s the value of the number of likes that post gathers? How can you estimate a real benefit of data generated using what more and more seems to be mostly fake accounts? Cambridge Analytica, anyone?
Some time ago, I had a talk with a friend of mine, about how much of a drop in the number of likes, followers, and shares would influencers suffer if a ban on the Chinese internet hit the world. How much like-farms determine the global numbers in terms of generating empty data? It’s not like I have an answer, to be clear, but it was just a very interesting thought experiment at that moment.
It would seem the whole world became some sort of a slave to its own suggestion: we should trust the numbers, the numbers are good, the numbers don’t lie. And we tend to forget where all those numbers come from.
We’re supposed to design FOR PEOPLE, not for data, though.
Especially since we focus on the most vocal groups of our users, and those tend to skew toward the negative — it’s more common for us to shout in anger than in happiness. Humans are truly feeble creatures, unstable in their beliefs — one moment they love your service, the second they hate you for what you did to “their” product. 1Password is one of the latest examples and its newest version is not even out yet (as of this writing), but people are already up in arms quitting the service left and right (or implying to do so) because “how dare you to change something”. It is really disheartening to read things like that, and I can’t imagine how much more it is troubling if it’s YOUR product people are hating on so much.
I think we put too much trust in all those tools and those numbers, but we let things go too far for anyone to point out that the emperor has no fucking clothes. We build our worlds around the idea it all has to work somehow, otherwise, it all might fall apart.
We allowed ourselves to be manipulated by data and statistics. Some of us build empires based on those numbers and keep exploiting other people for their personal gain in every corner of the world. People were digging holes in the system and creating opportunities for themselves since the dawn of mankind, but now access to knowledge is easier than ever and almost instant. Being “civilized” now only means we use computers instead of stone clubs. Recent developments surely are a sign of coming changes, I think, where we cannot depend on data alone. Surveys and polls show only a relative fraction of every situation.
Data in itself is not to be trusted.
I think we should remember to use it together with knowledge of how humans work in general, and how that relates to our users. Every system and application has a group that will be more vocal and willing to share their experience, especially the negative ones, but most of the time they will not represent the silent majority. We should listen to all of them, true, but we shouldn’t use the loudest input as the only way of defining the direction our businesses should follow.
