You're reading for free via Miklos Philips' Friend Link. Become a member to access the best of Medium.
Member-only story
AI and Design: why AI is your creative partner
AI and its subset disciplines, such as Machine Learning and Computer Vision are shaping the future. Design tools and designer roles, workflows, and processes will be molded by it. In light of this, we need to start thinking about AI not as “artificial” but as augmented intelligence, and in the ways we can take advantage in order for it to become our creative partner.
It’s time to shift our mindset from a human vs machine to a human plus machine mindset. Becoming familiar with AI will shift our approaches and ignite new thinking. AI in design will be more about designing awesome experiences, not UIs — creating better products, services, and improving people’s lives.
An analogy
Take an isolated tribe in the Amazon rainforest. These people have had no contact with civilization, and have never seen anyone who looked different from them. Imagine — completely cut-off from the 21st century. No TV or electricity, no phones, no Twitter, Instagram, or Netflix.
This is a photograph of the low-flying helicopter that due to a diversion because of a storm, took the National Geographic photographer Ricardo Stuckert out there, above the tribe. What did the helicopter look like to them? Scary? You bet.
In fact, they shot arrows at the helicopter! Terrified, to them, it seemed like a monstrous flying beast! Were they wondering: “is it a massive metallic dragonfly?”
Why am I bringing this up?
Because we fear that with which we are unfamiliar. The fear of the unknown.
Do we have the same response to AI?
Remember this guy?
A friendly dude from a famous Hollywood movie…
Wait! Maybe this is a better representation of him.
We’ve heard it all by now: AI is going to eliminate our jobs! The Machines are taking over! But will it really happen? Are we going to be terminated as designers/developers? If you do a Google search for, “Is AI going to…” you will see these autocomplete phrases in the dropdown: “Take over the world” “Take our jobs…” “Is AI going too far?”
Notice that these all have negative connotations. There isn’t one positive among these. How about “make our lives better,” or “help save the planet”! In other words, there is still a lot of fear around AI—mostly due to not being well-informed about it.
How about something positive and helpful? How about helping blind people “see objects” in their paths? They could wear a chest camera that would use computer vision and relay different sounds about their environment. Each type of sound would mean something different: a car, a bicycle, a person, a lamp-post, and where the pedestrian crossing is.
This can already be done today using YOLO, a state-of-the-art, real-time object detection system. In this scenario, AI is unlocking a myriad of possibilities for people with disabilities and promising unique ways of experiencing the world.
The machines are coming
But let’s bring it back to our daily work before we jump into more esoteric stuff. Things are rather slow. Tedious. A lot of manual work. Stand-ups. Meetings. Reviews. Decisions. User Research. Wireframing. Prototyping. More meetings! More Testing. More reviews. Iterating. Testing again. Blah-blah-blah. BORING! Surely there is something better, faster and more exciting!
There is.
Because the machines are coming — in a good way! AI is a collection of incredibly powerful, fast computer systems and algorithms. Machines that work and react like humans and give the appearance of human-like intelligence.
Machine Learning is a subset of AI; it enables computers to teach themselves to carry out tasks. (It’s more complicated, there’s supervised learning, unsupervised and semi-supervised learning, labeled and unlabeled data, structured and unstructured data. It’s all under Data Science. But let’s not go into that.)
To put it simply: Machine Learning is a system that receives inputs, produces outputs, then checks the outputs and adjusts the system’s original algorithms to produce even better outputs.
Machine learning is already used today in a wide variety of applications. For example email filtering in Gmail, personalized recommendations (just about any social media feed, Netflix…) voice-based interfaces (think Alexa, Siri, and Google Home); chatbots, computer vision applications such as face recognition, and Google classifying millions of images for you to search through.
In the next decade the AI revolution will filter through everything, and it is predicted by some that it will be a more dramatic shift in technology than the use of the personal computer.
A machine learning algorithm is about pattern recognition and learning cycles. This can be seen in action today. For example: based on a user’s demographic profile, preferences, and activity: their location, search, and browsing history we get personalization (Twitter, Google, Instagram, Netflix).
We also get Amazon recommending products to customers, Google Home and Amazon Echo talking to us, and Waymo navigating the world with self-driving cars. A common mistake people make is to assume machine learning is magic.
It isn’t. It’s just an amazing tool.
In contrast to machines — let’s look at the human brain. There is nothing like it in the known universe. It has 100 billion neurons and its processing power is 11 million bits of information per second processed simultaneously, granted, most of it unconsciously. We have the most powerful quantum computer in our heads! This is why we were able to overpower and eliminate this guy! 👇🏻
AI is narrow and very focused. AI is good at accomplishing pre-defined, specific tasks. All of those tasks have to be given to the system, they don’t operate on their own, independently. To quote Philip K Dick: Do androids dream of electric sheep?
Do machines have feelings, desires, and aspirations? Do machines have senses? A sense of smell, taste or touch? No. Those are all human qualities. Humans are endowed with a spirit and intelligence unmatched in the universe. AI doesn’t have the ability to contemplate the past, or imagine the future.
Let machines do what they do best: collecting, sorting and analyzing data, optimizing, pattern recognition and rapid learning. Let designers and developers do what they do best: creativity, insight, abstract thinking, making unusual and innovative connections between things. Creative people will take advantage of AI in ways seen never before because we can use the technology to augment us.
When it comes to product development, we need to consider AI not as “artificial” but as augmented intelligence. In fact, Ginni Rometty, the ex-CEO of IBM said that she preferred the term “augmented intelligence.” AI is not going to take our jobs, it’s going to change our jobs.
The US military is already working on “physical augmentation” projects. 👇🏻
We can think of AI, as our augmented intelligence that will amplify our capabilities. We’ll be freed up to engage product design in more creative ways because cognitive augmentation will reduce the time spent on slow, mundane tasks. When we recognize this, it will eliminate fear, anxiety, and all the silliness, and we will realize that AI is our friend. AI will simply augment our brain, amplify our creativity, and speed up the creative process.
Let’s see how.
AI can function as our assistant, do some heavy lifting with data collection and analysis. Finding patterns, making connections, and drawing conclusions. Sifting through reams and reams of surveys, interviews, observations, audio recordings, videos, and other user research data. Based on all the user data it receives, AI can even predict what design pattern will work best.
AI-enabled user research and testing will allow designers and developers to test designs at speeds never before possible. Think multivariate tests and A/B tests, testing emotional responses with facial recognition, and generating eye-tracking heat-maps with thousands of users in seconds. This significantly sped up process will help us emerge with actionable insights faster.
What if, early in our product design process we could quickly come up with all kinds of different UIs that could be tested by AI to arrive at the most optimal design? We could test designs against known usability standards, best practices, and conventions in a fraction of the time we have previously been used to.
☝🏻Let’s say we’re working on a B2B dashboard design. We feed the design into an AI-driven UI analysis tool. We run a bunch of tests against a certain set of usability standards, accessibility standards, and interaction design principles.
Here’s one of the screens from that set after we ran the tests. This is computer vision and machine learning at its best. Identifying patterns, analyzing, and flagging problem areas.
Looks like we have usability and accessibility issues with this design. We can make adjustments, and rerun the tests until the design is optimized. With AI, we can do this in a fraction of the time it used to take.
We can also generate design systems, component libraries, and styleguides in minutes. Our AI tools can generate the design system complete with ready-to-use code, and apply updates automatically so everyone on the team has the most up-to-date components at their fingertips. (I would bet that several companies are already working on such a tool.)
Airbnb already has an experimental system that incorporates computer vision — a discipline inside artificial intelligence — that generates finished UIs and code from sketches.
It’s a system that generates a prototype with blocks of UI designs from a set of predefined components in their library. Things are assembled on the fly, ready to go. Here it is:
Microsoft also has a computer vision system that product teams can use to speed up the design process. Sketch2Code uses computer vision and AI to convert drawings to working HTML prototypes.
The product team can shares ideas on a whiteboard. For example, take a picture of it, and it’s converted into code using computer vision and AI.
An important thing to point out: Remember what was said about “letting machines do what they do best and letting designers/developers do what they do best”?
Machine learning is like having millions of extra assistants, not one Einstein.
Machine learning does the grunt work, so product designers and developers can do more creative thinking. AI is meant to work with creativity, not replace it — it’s an important distinction. After all, we are going to be managing the input and grooming the output.
With fast UI design assembly by AI systems, we’ll be able to run simulations with real-looking prototypes. Not quite a finished product, but with AI, we’ll be able to put them in the hands of people quickly, test and learn. Then, iterate rapidly, continuously optimizing designs. This would actually unleash our creativity because we’ll have so much more time for creative experimentation.
Integrating machine learning into our products we can elevate UX by leveraging AI-driven suggestions for hyper-personalization. With this type of anticipatory design, we can accurately predict what a person will want to watch, read, or purchase.
The ability to do so will make it possible to deliver unique digital experiences for each person — and we are not far from it today. Hyper-personalization driven by AI will elevate UX on media sites, travel, news, social media, eCommerce, and more.
More and more tools to enable AI-assisted product design are coming onto the market. There are AI-driven user-testing tools such as Vempathy, an AI tool that employs emotion detection algorithms in product test recordings already out there.
It analyzes over 4,000 data points per minute, looking at facial expressions, listening for the tone of voice and sentiment, and it will show you exactly when people become confused, frustrated, or angry with the product.
Computer Vision
One of the most exciting applications of AI is computer vision.
From helping blind people “see objects,” to classifying images, to ID scanning and facial recognition for verification — computer vision is becoming more and more widespread. In fact, object detection — identifying the subject of an image — is already at 98% accuracy, and AI is now better at classifying images than humans.
Beyond watching babies and steering self-driving cars, many startups are already working on eCommerce apps using computer vision. Imagine taking a picture of someone on the street who’s outfit you really like. If you dare! Maybe a little weird… AI will analyze the image and in seconds give you the option to buy them. Amazon already has an experimental AI feature in its app that allows people to upload a picture or a screenshot and Amazon will then search its site for similar items.
More and more apps are coming out that use computer vision in clever ways.
👇🏻 This fitness app uses computer vision and deep learning to measure exercise speed and count steps, squats, push-ups and more just by using your mobile’s camera.
Various augmented reality apps are innovating where AI and AR work together, enabling hair color try-outs, virtual makeup, and checking out different nail colors. With this technology, people can also try on outfits, shoes, jewelry and more. This kind of AI and AR technology is coming — not only to apps and retail — but into every area of our lives.
Combining augmented reality with the power of AI employing deep neural network technology (a form of machine learning), you can check out what different hair colors would look like on you.
It’s an app called MODIFACE and they used 220 thousand hair images over 6 years to feed the machine data. Now the system can map any color onto any hair. 👇🏻
Practical implications
So, what will be the practical implications for designers and developers?
How are AI-driven systems going to help us? Two stages: creation and consumption.
At the creation stage: AI assisting designers and developers during the product development process. At the consumption stage: as people are using the product, AI delivers massively improved user experiences with hyper-personalization, anticipatory design, augmented reality, voice interfaces, computer vision, and more.
Our job as designers and developers is to use AI as an augmented system for what it does best, while we — humans — curate, test, and guide. Trust but verify. Have fallback systems if things go haywire. Test the heck out of it under all kinds of scenarios.
We need to define parameters and put up guardrails, especially when a bad output could cause harm, such as during facial recognition, health care applications, mental health applications, and more. We can’t let AI go out there like a loose cannon.
There will be glitches! At one point Amazon’s facial recognition system linked the faces of 28 members of the United States Congress to criminal mugshots. Talk about #AIFAIL!
This emphasizes the importance for designers and developers to guide, curate, test and verify AI’s output. Because AI is a tool that amplifies intent, we must be careful and consider possibly harmful outcomes. We must have strong AI ethics and have a strong set of AI-driven design principles that will guide us.
AI-Driven Design Principles
We must acknowledge that all models made by humans — reflect human biases so there is a need for principles.
- Designing for trust means being transparent and sincere with everything concerning data.
- Design for humanity means humanizing experiences with feedback, language, and tone.
- Designing for less choice means using anticipatory design and removing unnecessary decisions.
- Designing for minimal input means solving significant user problems with minimal input expected from them.
- Design for no discrimination means that as we build models for machine learning we must remove all possible forms of discrimination. Outcomes must be lawful, ethical, and robust — both from a technical perspective and the social environment.
To stay ahead of the game, designers and developers need to start learning about AI, ML, AR, voice user interfaces, and computer vision. Start exploring the capabilities of AI and how to best use this new technology. It would also be a good idea to start diving into the AI-driven design principles mentioned above.
The best thing you can do for your career now is to start learning about these new technologies at both the development stage and for better user experiences in the product itself.
The future is clear. It’s time to shift from a human vs. machine to a human plus machine mindset. Over the next decade, the AI revolution will be an evolutionary phase that will require designers and developers to adapt or risk becoming irrelevant. Those who learn to work with the new AI technologies will come out ahead. Those who don’t may be left behind.
It is crucial that we maintain a human-centered approach to crafting AI-driven experiences. “AI” — as in AUGMENTED Intelligence — is going to be a creative partner that can help us accelerate our creativity, and explore design possibilities more deeply than ever before.
AI will not take over our work, it will change the way we work.
Don’t fear AI, embrace it. Because AI and you will make you a better designer/developer.
👋🏻 Hello! Thanks for reading and getting to the end of the article. 🙂
I’m now available for starting people on their UX designer careers through my UX Course or for ongoing mentorship on MentorCruise.
Learn UX with me at half the cost of crappy bootcamps. Get certified & build a portfolio for a high-paying job. 7-day free trial http://uxwithmiklos.com
