Designing with AI: ‘prompts’ are the new design tool

Exploring AI image generation.

The prompt

sci-fi dream autumn landscape with a white castle, colorful autumn maple leaves, trees, satellite view, hyper detailed, dreamy, volumetric lighting

The image

Midjourney prompt created using www.promptmakr.com

I am sure by now you have heard about AI image generators like Midjourney, Dall-E, Stable Diffusion, Nightcafe etc. Irrespective of which tool you use, what’s surprising is the fact that now you can give simple text instructions to a machine and it will generate some amazing images for you like the example above.

But you must also be wondering, then why is it that not everyone is able to produce these stunning images we see from others on Midjourney showcase or Instagram.

Introducing “Prompt Engineering”

The answer lies in what is known as — Prompt Engineering. Prompt Engineering is the concept in Natural Language Processing (NLP). Usually machine learning models are trained using millions and millions of data points and they start forming their own knowledge base, so one can never really figure out what the machine has learned. The best way to know what the machine knows is to craft different prompts and see what output we get.

The art of generating desired images through these Text-to-Image generators lies in gaining a good understanding of this knowledge. This is where Prompt Engineers come into the picture.

Prompt Engineers are people who have mastered the art of writing prompts that give consistent good results and leverage most of the features provided by the image generator engines. In this article we will focus mainly on Midjourney prompt engineering and learn the nuances of writing good prompts as well as some tricks to generating stunning images.

Basic prompts

Let’s look at two prompts and the images they generated.

Left image:

side profile car

Right image:

industrial design side profile sketch of a futuristic car, illustration, copic marker style, modern vehicle, creative