The Fluid approach to UX walkthroughs examines both heuristics and cognitive evaluations at the same time. By combining these two approaches, an evaluation gains benefits beyond just one of the evaluations in isolation.

Our aim for Fluid, "Software that works - for everyone", takes in accessibility as well as usability. Rather than having two separate inspections, we want to have a unified inspection that addresses both areas.

Overview of the Approach

When evaluating  a system using the Fluid approach, you want to keep both usability (aka. the heuristics) and accessibility (aka. cognitive concerns) in mind. A good way of thinking of the approach would be:

By answering these two questions at any given moment of your evaluation, you are evaluating both the big picture and specific details at the same time.

Before you Begin: Choose your Persona and Establish your User

While traveling through a service performing the evaluation, a cognitive walkthrough is performed through a step-by-step exploration of a page to see how well a particular type of user will be able to accomplish his or her objectives. The user in mind is represented by a persona, or based on a profile of a known user demographic.

1. Choose a persona

If you do not have a persona created, but have access to real world users of the system or demographic information of those users, you can still create a partial persona (see Personas for more information on how to create and apply personas).

Your choice of persona(s) should be documented in your evaluation.

We strongly recommend creating a persona based on real-world users, even if the persona is partially completed, as this will likely give more accurate and revealing results.

2. Specify an explicit user goal

Note that separate walkthroughs may be needed for each persona, although some issues will likely show up in more than one walkthrough, resulting in later walkthroughs going more quickly than earlier ones.

Fluid UX Walkthrough Checklist

Checklist: Heuristic Principles for Usability 1

To conduct an heuristic evaluation of a service offered through the web, travel through the pages of the site, reflecting on each of the listed principles, and recording compliance and violations. The heuristic principles should also be kept in mind during cognitive walkthroughs.

___ Visibility of system status
The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.

___ Match between system and the real world
The system should speak the user's language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.

___ User control and freedom
Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.

___ Consistency and standards
Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.

___ Error prevention
Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.

___ Recognition rather than recall
Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.

___ Flexibility and efficiency of use
Accelerators - unseen by the novice user - may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.

___ Aesthetic and minimalist design
Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.

___ Help users recognize, diagnose, and recover from errors
Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.

___ Help and documentation
Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.

1 The above section lists the original Nielsen and Molich heuristics, as refined by Nielsen.

Protocol: Cognitive Walkthrough

The following protocol is performed using

Work out the sequence of steps the user should go through, to accomplish the goal.

For each step, ask the following questions:

1. Will the user know what to do at this step? Is complex problem solving needed to figure out what to do?

2. If the user does the right thing, will they know that they did the right thing, and are making progress towards their goal? Is complex problem solving needed to interpret the feedback?

Addressing Accessibility in a Cognitive Walkthrough

An accessibility walkthrough follows procedure similar to the cognitive walkthrough described above, with the difference that the user has one of a number of disabilities such as low vision, blindness, impaired hearing, motor control limitations, or cognitive issues.

In an accessibility walkthrough, the main consideration is how these limitations affect the use of websites or software. For example: blind persons and some persons with limited motor control need keyboard-only operation; some persons with cognitive issues need visuals that reinforce text; persons with low vision must enlarge page content; deaf people require video captioning and visual, rather than auditory, prompts. You also must consider the assistive technology the user will use. Examples include screen magnifiers such as ZoomText, screen readers such as JAWS, or combination screen readers/enlargers such as Kurzweil 3000.

Before proceeding with the formal walkthrough, it is useful to perform the following steps:

1. Assess the overall layout and structure of each page.
2. Play around with the layout: enlarge the font size; change the size of the window (bigger and smaller); adjust the resolution.
3. Use the Tab key to traverse the entire page.

To perform an accessibility walkthrough, adopt a persona, select a goal, and perform the steps necessary to accomplish the goal. At each step, ask the following questions:

1. Will the user know what to do at this step?

  • Is complex problem solving needed to figure out what to do?
  • Are the cues provided accessible to the user?
  • Can the cues be understood by someone who doesn't process text well?
  • Can the cues be found by someone who can't scan the screen easily?

2. Will the user be able to carry out the required action?

  • Can it be performed easily by a keyboard-only persona?
  • Can it be performed without visual monitoring?

3. If the user does the right thing, will they know that they did the right thing, and are making progress towards their goal?

  • Is complex problem solving needed to interpret the feedback?
  • Is the feedback accessible to the user, and can they find it (as for cues)?

To conduct an actual accessibility walkthrough and assessment, it is recommended that the reviewer select a persona to adopt and then follow one the detailed protocols listed in the section on heuristics (above):

  • Simple Accessibility Walkthrough Protocol
  • Comprehensive Accessibility Review Protocol for PC
  • Comprehensive Accessibility Evaluation Protocol for Macintosh

Reference

More detail on each of the heuristic principles can be found in be found in an expanded list from Deniese Pierotti of Xerox, which itemizes specific things to look for when evaluating a system with Nielsen's Heuristics. See Heuristic Evaluation - A System Checklist.

The Fluid UX Accessibility Working Group has created a set of protocols for assessing accessibility:

A paper from Claire Paddison and Paul Englefield provides a list of nine heuristic principles for accessibility evaluations:

Paddison and Englefield include in their paper a general discussion of the heuristic approach. This is recommended reading for all reviewers.