Testing readability tests: Flesch–Kincaid, ARI, and Gunning Fog

The lousy powers behind Hemingway Editor, Grammarly, and Readable.com.

The Cure of Folly by Hieronymus Bosch (c.1494 — c.1516, PUBLIC DOMAIN)

Table of Contents

• Intro
• Writing for everyone on the bus
• Anti-history of Flesch-Kincaid, ARI, Gunning Fog
• How do readability tests work?
• Testing readability tests
• Checking the text with FK
• Checking the text with ARI
• Checking the text with GF
• Final results
• What makes readability tests lousy?

Intro

At the core of the most popular online editors are readability tests that were invented 60–80 years ago.

These UX writing and copywriting platforms use Flesch-Kincaid tests, the Gunning Fog Index, or Automated Readability Index to measure if the text is easy to read:

• Readable.com
• Hemingway Editor
• Writer.com
• Grammarly

Flesch-Kincaid tests (FK or FKT) and the Gunning Fog Index (GF or GFI), why do they sound like something from particle physics? Why Hemingway Editor prefers the Automated Readability Index (ARI) to every other readability test?

Most importantly, are these tests even viable in 2023?

Let’s see what I’ve dug up.

Writing for everyone on the bus

The Bus by Frida Kahlo (1929, FAIR USE for non-profit educational purpose)

My goal is to write in a simple and accessible language. This is my mission. I stupify the text for people who no longer read. This doesn’t mean I think my target audience is stupid. It’s just that they may not have all the time in the world to read something that requires extra effort.

Readability tests help me evaluate if my writing is really for everyone (as law professors say, “for everyone in the Clapham omnibus”). The goal of this article is to check how reliable these tests…