User experience, CLIs, and breaking the world
Starting at the end of last year, we put a plan in motion to revamp the CLI for the IBM Cloud Kubernetes Service. We focused on improving the user experience and overhauling the code. The key is to break compatibility, but surprise no one.
We had several problems with the CLI after a few years of adding new features. They ranged from user experience issues to technical debt buildup:
- Overwhelming help output. If you didn’t already know what you were looking for, finding the right command was a daunting task. There were only two forms of help: list all possible commands or show one command in detail. For smaller CLIs, listing all commands isn’t bad — but when there’s 100+ commands, it’s a non-starter. No one wants to order food from a menu the length of a dictionary!
- Inconsistent names and behavior. Writing consistent code as a single developer is easy — no conflicting ideas when there’s only one developer’s opinion! However, larger teams naturally create small changes in behavior.
Some flags were written in--camelCaseand some--with-dashes, logical groups of commands didn’t exist, usage text was different, and some arguments were both positional and flag arguments. - Unclear separation of concerns. A single command’s code was very difficult to reason with in isolation. Altering one command required changes in several seemingly-unrelated places. The list of flags and the command implementation lived in completely different locations.
These issues came to a head when a new set of commands pushed the help text way off the end of our monitors. Something had to be done, but what? We designed and built some high-fidelity mockups to try new approaches with our users to see what sticks.
User testing
A consistent “language” in a CLI is key for intuitive interaction. Two common forms of command structures came up in our research: noun-first and verb-first. We wanted strong feedback on these structures, so we put together a test where users could perform their daily work with the new commands. Sometimes testing the full experience right away isn’t an option, so we used a high-fidelity mockup to determine our users’ preferred CLI structure.
To run the test, users installed a Python script as a wrapper for the CLI. It allowed users to flip between noun-first, verb-first, and the original structure. The wrapper read CLI arguments and flags, translated them to the original command, then ran it.
This made the mockup significantly easier for our testers to adopt into their daily flow, and they got a great feel for which variant they liked most. Based on our research and user testing results, we chose grouped noun-first commands.
Nouns? Verbs? What’s going on?
An intuitive interface has a consistent language for completing tasks. A noun-first CLI may have commands like dog run, dog walk, or even dog ball fetch. A verb-first CLI may use run dog, walk dog, or fetch dog-ball. Though verb-first seems strange, it can form a very strong pattern for nouns that have very similar actions. Another variant is “grouped” commands where visually separate groups of commands help locate the desired action.
Further investigation on these structures also revealed maintenance concerns. If the CLI’s nouns will always support all verbs, then verb-first is easy to scale and add new nouns in the future. On the other hand, if nouns have vastly different actions from each other, then noun-first might be a better choice for a low-maintenance CLI.
While testing the mockup, users strongly favored both noun- and verb-first variants over the old commands but liked them roughly equally. A few users preferred verb-first more since it aligned with kubectl, the Kubernetes CLI, but ultimately both variants worked well.
Results
User testing results strongly indicated we could adopt either noun-first or verb-first, but the old command structure is clearly worse. The tests showed grouping commands together made it easier to find them, and research showed us which command structures would be best for our CLI’s long-term maintenance. Since our commands are primarily nouns with very different verbs, we decided to move forward with grouped noun-first commands.
Building a better CLI
Fixing all of the CLI’s issues would be great, but the work can’t happen overnight and it’s wise to avoid total rewrites. A refactor with carefully planned migration steps is a good way to improve over the long haul.
Backward compatibility in a CLI is also very important — without it, automated scripts would break and every change would only make things more difficult for our users. Not only did we avoid breaking compatibility, we had to run user testing and code migration at the same time. This would be tricky and probably take a while, so we put forward a flexible 4-step migration plan.
Retrofit structured commands
Our CLI was not originally designed to support more than one layer of commands. Some rework was needed to support structured commands.
However, the upgrade was too massive to apply all at once. In this phase, we created a new framework to enable small refactors, like moving only a few commands at a time. Eventually all commands would support both structured and flattened commands.
Our new framework needed to be easy to adopt in code and easy to try out new features right away. We added an end-to-end test fixture to reduce test code churn during adoption, a beta version environment variable (IKS_BETA_VERSION=0.4) to assist user testing, and a translation utility to register both flat and structured commands from one spec.
Enable both structured and flattened commands
Migration kickoff! Releases for this version only showed help for flat commands, but allowed beta testers to enable structured commands. Structured commands also enable “structured help” that displays only the immediate subcommands, rather than all of them at once. User testing provided invaluable feedback and heavily influenced design changes as we went.
We moved only a few commands at a time to the new framework, simultaneously converting unit tests to a more portable end-to-end test format. Commands were moved into isolated modules where both command registration (e.g. names, flags) and action code sat together.
The code churn in this phase was immense and definitely took the most time to complete. The version control diff was massive: it came out to 33k new lines and 26k deleted, and the code churn diffs (total adds/deletes across individual pull requests) was approximately 73k added and 50k deleted. These changes were fairly taxing on us, but we were confident in our plan and pushed through it.
Structured help, start the countdown
The third phase released structured commands while keeping backward compatibility. Now, users can immediately reap the benefits of “eventual disclosure” and avoid the wall of text. They’re also made aware of the countdown to a release containing breaking changes. This release includes warnings and a “script update” command to encourage users to stop using the old, hidden commands and use structured ones instead. The ibmcloud ks script update command changes shell scripts to use the new commands, automatically.
Break the world (but not really)
Our last phase of the refactor is to release breaking changes, rectifying the last pieces of the old experience. By this point, the warnings and tools have provided sufficient notice of breaking changes, so a full rollout is very low risk to our customers.
Breaking the world
Backward compatibility is the name of the game for CLIs and APIs alike. The key is to break the world but always run user tests, provide tools, and leave plenty of time to make a smooth transition. With our transition complete, users can now intuitively understand and easily navigate the CLI.
Try out the new CLI
You can install the CLI by following this tutorial or, if you already have it installed, run ibmcloud update kubernetes-service to get the newest version. For more info on the changes we released, you can read more on the IBM blog or on the CLI changelog.
If you have questions (or suggestions), engage our team via Slack by registering here and join the discussion in the #general channel on our public IBM Cloud Kubernetes Service Slack.
Shout out to Chris Kirkland and Lucas Copi for their significant contributions to the CLI’s new design and overhaul. 👏
Disclaimer: I work at IBM. The views and opinions expressed here are my own and do not necessarily represent IBM’s policies or positions.
