Introduction

This document provides background information on performing a UX Walkthrough. Detailed directions for performing an evaluation on a particular system can be found on the appropriate UX Walkthrough Report Template.

Goals

We want to develop a kind of combination of heuristic evaluation and walkthrough. Heuristic evaluation is done by examining the interface to a system as whole, and hence isn't specific to any particular user task. Walkthroughs are done by tracing the user actions, and associated cues and feedback, for one or more particular tasks, and as such, don't attempt to cover the entire interface for a nontrivial system. On the other hand, a walkthrough can pick up issues in working through a task that can be hard to detect when examining the interface as a whole, relating especially to the adequacy of cues and feedback in context.

Our aim for Fluid, "Software that works - for everyone", takes in accessibility as well as usability. Rather than having two separate inspections, we want to have a unified inspection that addresses both areas, if we can.

Choosing what to examine

Within a product, we want to examine tasks and subtasks that use parts of the product that have some functional affinity, that is, parts that in some abstract sense "do the same kind of thing". We believe this will help us identify especially useful components, ones that can be used in more than one place in a product. For example, the Lightbox component can be used to do lots of different things. Abstractly, it's a component for "rearranging the layout of things".

By coordinating our choices across products, we can promote the same goal on a wider scale. By pooling our findings about how related functionality is used in different products, as well as in different places within the same product, we can design components that can be reused as widely as possible.

Since uPortal, Moodle and Sakai are all large and complex products in themselves, we will do the UI walkthroughs in stages, focusing on subsets of activities the products support in each phase.  For example, the first phase may focus on content management related activities.  By focusing the product evaluations on similar user activities, we will be able to understand user needs and the interactions that fit those needs across products.  The goal is that UI components can be designed and developed to fit the specifics of each product.   For example we may find that organizing content is cumbersome across the products.  A generic version of the "Lightbox" component, the "Organizer" could meet a broad set of needs.  As always, the context of an interaction shapes the specific implementation of a interaction so "Organize" could be the umbrella component with several specific implementations that meet the needs of different kinds of content and/or contexts.  Content management is being loosely defined as authoring, maintaining, organizing, viewing and sharing content - in this case digital content.  One can imagine the many cross-cutting interactions that are "componentizable" around managing content. 

Note that this is not a completely "top down" approach to our priorities, in that we are not starting directly from a determination of what tasks are most important for users (though we an presume that functions that crop up again and again in our products are important to users). After we are well launched in discovering components in this way, we may want to allocate effort to look at the user experience for the products more holistically, to assess what we may be missing in taking the functional view.

Procedure

1. Preparation

Obtain working software, a prototype, for the system you intend to inspect. We will probably also want to do inspections from design documents from systems not yet implemented, but that case is not addressed (yet) in these notes.

• For the walkthrough portion of the inspection, choose one or more representative tasks to inspect.
• If you are also doing an accessibility inspection, btain an appropriate tool to use in inspecting the user experience for screen reader users. (Query: what tools have people used for this?)
  • JAWS is a widely used screen reader - a free time-limited demo is available.
  • Fangs is a Mozilla Firefox extension that displays a text representation of a web page in a manner intended to be similar to how a screen reader would read it.

2. Process

Plan to inspect the system from four viewpoints: typical users, screenreader users, keyboard-only users, and users who have difficulty processing text.

(Query: Should we also inspect for low vision users who are not screenreader users? Other cognitive disabilities not captured by difficulty with text?)

You can decide how you want to organize these inspections, for example by doing them sequentially or by examining a portion of the system from each point of view before moving on to another portion. Be sure to share your experience with these different approaches.

There are also different ways to organize the heuristic analysis and walkthrough aspects of the inspection. Here is one approach that combines some of the prep for the walkthrough with part of the heuristic analysis.

The walkthrough method requires that the analyst know what the right way to perform a task is before doing the walkthrough itself for that task. So, as the first part of your inspection, examine the interface with the dual aim of checking the points listed in the Heuristics Checklist below AND working out a good approach to your task or tasks. (You can decide whether you want to pick one of your tasks and complete all work on that before beginning another, or do the prep for all tasks before doing the walkthroughs on them.) Make notes of any issues you encounter in figuring out how to do the tasks, as well as any other problems you identify in this phase using the heuristics.

Once you have worked out how to perform one or more tasks for which you will do a walkthrough, do the walkthrough. This means working through an appropriate method for the task, step by step, checking the points in the Walkthrough section of the Heuristics checklist below. Keep carefully in mind that the walkthrough is NOT a user test, with you being the user. That's why you START the walkthrough already knowing a good way to perform the task. At each step, you are checking whether SOMEONE ELSE, without benefit of your prior knowledge, will be able to figure out what to do, do it, and respond appropriately to the result of the step.

Remember that you want to inspect not only for typical users, but also for screenreader users and others. This means that you need to consider the Walkthrough points for ALL of these users:  an interface may provide adequate cues for some step for typical users, but the cues may simply not be available to screen reader users, or may be hard to interpret and process for users for whom text processing is difficult. Similarly, required actions may be easy for mouse users, but difficult or impossible for people who do not use a mouse.

Once you have completed all of the walkthroughs you plan to do, keep in mind that Heuristic Evaluation includes surveying, as well as you can, ALL aspects of the interface, not just those that figure in particular tasks. Do what you can to explore the interface broadly, getting (and recording) a feel for any issues triggered by the Heuristics checklist.

Inspecting widgets. While our inspections are intended to provide useful feedback on the overall user experience provided by the systems we work on, Fluid has a particular focus on developing a usable and accessible widget set. This means we need to use special care in inspecting widgets. Because widgets are intended to be widely reused, special treatment for widgets in our inspections can also save us work: if someone inspection system A has already inspected a widget, someone inspecting some system B that uses the same widget doesn't need to repeat the detail work.

To realize this potential work savings, while at the same time making sure we provide careful inspections of all widgets, we should (a) make a list of all the widgets we encounter in an inspection, (b) check to see if an inspection for this widget is already available, (c) if it is, don't take the time for an in-depth inspection of the operation of the widget, and (d) if it is not, do a careful inspection, including accessibility aspects, and record the results in a shared list of widget inspections (Note: we need to create this shared list, which includes deciding how we are going to identify and describe them.)

An example of the application of this idea would be encountering a multiple selection widget while examining an interface overall, or doing a walkthrough for a task. If the shared list shows that the widget has already been inspected, we would pass over it, assuming that the user can make it work (or, more accurately, assuming that any problems have already been found and recorded). If it has not been inspected, we would do the inspection, carefully. For example, we would determine whether the multiple selection can be operated without the mouse; in fact this is not always possible with existing multiple selection widgets.

3. Recording results

While doing the inspection, you should note any issues you find, which heuristic checklist items are violated, and describe where in the system the problem occurs. We'll need to develop a system for giving these descriptions, and sharing the results.