Published on 28 April 2017 by

Two words: Design. Technology. What do you think of when I say them? An iPhone? Your favorite web app? A Maserati?

Did you think of a programming language? Not many people do. But programming languages require design, too.

No one is ever surprised to learn that members of our user experience design team focus on the Puppet Enterprise (PE) console, the product’s graphical user interface (GUI). The surprise comes when they learn that we also focus on the command-line interface (CLI) and APIs.

As a user experience (UX) team, we care about all the ways that Puppet users interact with the product. Many people use Puppet at the command line exclusively. If we’re not thinking about that aspect of the product, we’re ignoring a major part of the user experience.

Fortunately, many of the same tools and techniques that apply to GUI design work for the CLI as well. Some techniques, though, lend themselves particularly well to CLI research and design. One that I rely on is called language norming. My former colleague Laura Chessman introduced language norming at The MathWorks, and some of us who worked with her there continue to use the technique and teach others about it. I use it whenever I want to see what particular words imply to users, like recently, when we were developing high availability (HA) for PE.

With the advent of HA, the development team added a set of command line actions that let users control a primary master replica, which keeps Puppet runs going when the primary master isn't available. HA provides users with the ability to provision, enable, forget, and promote a primary master replica. Provisioning designates a machine as the replica; enabling makes it functional; forgetting removes that functionality; and promoting makes the replica the new primary master. We weren’t sure, though, what the right subcommand would be for this set of HA actions. We had a few ideas, but lacked general agreement. Our top two contenders were puppet enterprise and puppet infrastructure. I decided to conduct a language norming study to see how these phrases resonated with users, and if they could lead us to a phrase that was even better.

A language norming study takes a predictable form:

  • A scenario lays out the context where the wording in question will live.
  • Question 1 is open-ended, asking the respondent what words she would expect to see, given the scenario.
  • Question 2 offers a set of possible wording choices and asks the respondent to rank these — along with her response to Question 1 — on two scales: one ranking the memorability of each choice, and one ranking the clarity of each choice.
  • Question 3 is another open-ended one. Now that you’ve thought about a variety of possibilities, which is the best?

Here’s how each of those pieces looked for our HA study, which was conducted in the fall of 2016, before the feature launched:

Background

In a future version of Puppet Enterprise, you will be able to provision a "primary master replica," which will include copies of many of the Puppet services, like PuppetDB, Puppet Server, and role-based access control (RBAC). If the primary master fails, the primary master replica automatically takes over to ensure that Puppet runs continue.

Scenario

In an effort to make your Puppet implementation highly available, you have provisioned a primary master replica. You will need to enable this replica via a manual step at the command line. Before doing that, you must determine if the services on the primary master replica are in a condition that will prevent you from enabling it successfully — for example, if there are services experiencing errors or warnings, or if replication is still in progress.

Question 1

In the absence of documentation or help, what command do you expect to type at the command line to see the relevant information about all Puppet services on the primary master, primary master replica, and compile masters, if you have any?

Question 2 (two parts)

Question 2, part 1

Question 2, part 2

Question 3

Question 3

My main source of survey responses was PuppetConf 2016 attendees. At the conference, the UX team set up laptops with various studies to showcase our work and to get feedback in a forum that offered us access to lots of current and potential Puppet users.

In all, 91 people responded to the language norming questions. As is often the case with this kind of study, there wasn’t a clear preference. Rather, a bit more than one third of respondents preferred puppet status, and another third preferred either puppet infrastructure status or puppet infra status. There wasn't a consensus among the remaining 21 responses.

  • People who preferred puppet status cited its simplicity as the reason for their preference.
  • People who preferred puppet infrastructure status preferred the clarity and specificity of the language. A few mentioned that the word infrastructure is long and difficult to type. This led three responders to propose puppet infra status as a compromise.

Although puppet status got the most votes, the team decided to use puppet infrastructure status (with the alias infra) for the release. The specificity of the language makes this choice easier to explain, especially in the absence of other information that would naturally fall under a puppet status command, such as agent status.

The study helped us understand what people liked and disliked about the various possibilities so that, when we did make a choice, we were better equipped to defend and explain it.

Language norming also helps us design better GUIs. Especially when there’s limited real estate — like on a button — choosing the right words is essential. This kind of survey gives us the data to understand what different words mean to people, so we can narrow in on the best choice.

Lori Landesman is manager of UX research at Puppet.

Learn more

Share via:
Posted in:
Tagged:
The content of this field is kept private and will not be shown publicly.

Restricted HTML

  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd> <h2 id> <h3 id> <h4 id> <h5 id> <h6 id>
  • Lines and paragraphs break automatically.
  • Web page addresses and email addresses turn into links automatically.