Checklist: Heuristic Principles for Usability ¹

A heuristic evaluation is an expert review of a user interface with reference to explicit principles termed "heuristics." During the evaluation, reviewers see how well the service fulfills the basic requirements for usability or accessibility.

To conduct a heuristic evaluation of a website or application, travel through the pages of the site, examining each one while reflecting on each of the heuristic principles. Make a record of where you find compliance and where you find violations. Note that a heuristic evaluation may also be done in combination with a cognitive walkthrough.

Heuristic Principles

___ Visibility of system status
The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.

___ Match between system and the real world
The system should speak the user's language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.

___ User control and freedom
Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.

___ Consistency and standards
Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.

___ Error prevention
Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.

___ Recognition rather than recall
Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.

___ Flexibility and efficiency of use
Accelerators - unseen by the novice user - may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.

___ Aesthetic and minimalist design
Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.

___ Help users recognize, diagnose, and recover from errors
Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.

___ Help and documentation
Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.

¹ The above section lists the original Nielsen and Molich heuristics, as refined by Nielsen.

Checklist: Heuristic Principles for Accessibility

The following is an abridged version of the Simple Accessibility Review Protocol. You may want to read the full version to gain a better understanding of how the principles apply to accessibility.

___ Assess the overall layout, structure and content of the page

• Is the page structured into logical sections?
• Are those sections clearly labeled, and is their function apparent?
  • Do paragraphs, sections, and tables have meaningful headings?
  • Is the information in forms grouped in relevant "clumps"?
• Is site content helpful to users of assistive technology?
  • Can headings, captions and links be understood independently of their surrounding context?
  • Does sentence structure follow writing conventions in a consistent way?
• Are there sufficient non-visual cues to convey the overall structure? (I.e., Are there sufficient text and/or explanations on the page to understand its purpose and functionality without relying on graphic elements).
• Are there sufficient visual cues to support page content?
  • Are visual cues recognizable without being differentiated by color?
  • Does there seem to be sufficient contrast between foreground (text) and background colors?
• Are the most important information and navigational elements prioritized high on the page?
• Is navigation consistent from page to page? Is the structure of content consistent from page to page?

___ Play around with the layout: enlarge the font size; change the size of the window (bigger and smaller); adjust your resolution

• Is all the text still visible? Does it overlap?
• Does visual context break down at large text sizes?
• Are headers, labels, and visual cues still correctly associated with their content?
• Do columns shift or realign correctly?

___ Use the Tab key to navigate through the entire page.

• Do all links and controls receive focus as you tab through the page?
• Can all controls be activated with the Enter or Space keys?
  • Is movement possible using arrow keys within the object (for example, to move around cells within a table, or within a form)?
• Does tabbing follow a logical sequence through the page?
• Is content easily accessible and not secondary to navigational "chrome"? Are navigation and content formats consistent from page to page?
• Are there any areas where you get stuck and have to use the mouse?
• Are shortcuts or skip links provided to make keyboard access quicker? Are they visible and clear?

___ Use Internet Explorer (6 or 7) or Firefox with Popup Alt Attributes Extension to check for alternative text for all images and title text for links.

• Do images have text alternatives for visual content?
• Are descriptions short (around 20 characters maximum) and using the alt attribute?
• Are decorative or meaningless images such as spacer graphics using empty alt text (alt="") so they will be ignored by screen readers?
• If a description cannot be adequately provided with 20 characters, does a graphic have a supplemental longdesc attribute?
• Is title text specified for links? When link text cannot be made descriptive enough by itself, more of an explanation can be provided through the use of the title attribute.

If you wish to do a more comprehensive accessibility inspection than is described here, you will find detailed instructions at the following links:

• Comprehensive Accessibility Review Protocol for PC
• Comprehensive Accessibility Review Protocol for Macintosh

Protocol: Cognitive Walkthrough

Sometimes performed at the same time as the heuristic evaluation (see left column), a cognitive walkthrough is a step-by-step exploration of a user interface to see how well a particular type of user (sometimes represented by a persona) will be able to perform a task or set of tasks.

___ Choose a user

• Choose a user from whose perspective you will perform the walkthrough. This may be easiest to do by selecting a persona - see UX Walkthrough Preparation and Execution for more information on this.
• If using a persona, ensure that the persona:
  • is adequate to judge what knowledge the user may plausibly be expected to have
  • specifies the particular needs, preferences, and limitations the user may have
• Note: separate walkthroughs may be needed for each persona or user type, although some issues will likely show up in more than one walkthrough, resulting in later walkthroughs going more quickly than earlier ones.

___ Define a goal and a set of tasks

• Determine the specific result desired by the user and motivating the interaction
• Define a set of tasks required to a accomplish the goal
• Lay out the sequence of steps the user/persona should go through to perform the tasks and achieve the goal. (Information on defining goals can be found in UX Walkthrough Preparation and Execution).

___ Perform the tasks

• For each step in accomplishing the tasks, ask:
  1. Will the user know what to do at this step? Is complex problem solving needed to figure out what to do?
  2. If the user does the right thing, will they know that they did the right thing, and are making progress towards their goal? Is complex problem solving needed to interpret the feedback?

___ Record results

Protocol: Cognitive Walkthrough with Accessibility

In an accessibility walkthrough, the main consideration is how low vision, blindness, impaired hearing, motor control limitations, or cognitive issues affect the use of a website or application. The cognitive walkthrough protocol is followed, with an emphasis on accessibility cues.

___ Choose a user

• Choose a user from whose perspective you will perform the walkthrough. This may be easiest to do by selecting a persona - see UX Walkthrough Preparation and Execution for more information on this.
• If using a persona, ensure that the persona:
  • is adequate to judge what knowledge the user may plausibly be expected to have
  • specifies the particular needs, preferences, and limitations the user may have
• Note: separate walkthroughs may be needed for each persona or user type, although some issues will likely show up in more than one walkthrough, resulting in later walkthroughs going more quickly than earlier ones.

___ Define the goal & tasks

• Determine the specific result desired by the user and motivating the interaction
• Lay out the sequence of steps the user/persona should go through to accomplish their goal. (Information on defining goals can be found in UX Walkthrough Preparation and Execution).

___ Initial Assessment

1. Assess the overall layout and structure of each page.
2. Play around with the layout: enlarge the font size; change the size of the window (bigger and smaller); adjust the resolution.
3. Use the Tab key to traverse the entire page.

___ Perform the tasks

    For each step in accomplishing the task, ask:

1. Will the user know what to do at this step?
   • Is complex problem solving needed to figure out what to do?
   • Are the cues provided accessible to the user?
   • Can the cues be understood by someone who doesn't process text well?
   • Can the cues be found by someone who can't scan the screen easily?
2. Will the user be able to carry out the required action?
   • Can it be performed easily by a keyboard-only persona?
   • Can it be performed without visual monitoring?
3. If the user does the right thing, will they know that they did the right thing, and are making progress towards their goal?
   • Is complex problem solving needed to interpret the feedback?
   • Is the feedback accessible to the user, and can they find it (as for cues)?

    Considerations:

• Blind persons and some persons with limited motor control need keyboard-only operation
• Persons with cognitive issues need visuals that reinforce text
• Persons with low vision must enlarge page content; deaf people require video captioning and visual, rather than auditory, prompts.
• You also must consider the assistive technology the user will use. Examples include screen magnifiers such as ZoomText, screen readers such as JAWS, or combination screen readers/enlargers such as the Kurzweil 3000.

___ Record results

Websites and web applications have the opportunity to provide additional information to persons with disabilities to help them understand and navigate more easily. Some of this information is visible to all users and can enhance their experience. Some of it is transparent to sighted users, but provides context for persons with visual, auditory or motor limitations.

In an accessibility markup (code) review, the focus of the inspection is not the interface as the user sees it, but an under-the-covers examination of the code used to implement it – often HTML. The person conducting the review must be familiar with the implementation language and know what standards of quality and correctness apply. Code reviews are an important part of an accessibility assessment since there are a number of established conventions that must be followed.

Under-the-covers Questions

• Is content separated from presentation through CSS?
  • Are buttons and other controls text-based?
  • Are tables used for data and not layout?
  • Does page content linearize?
• Do pages have frames?
  • Are they necessary? If so, do they have meaningful titles?
• Are headings hierarchical?
  • Is punctuation provided in headings to control screen readers?
• Are tables properly structured?
  • Do they contain headers and summaries?
  • Are table cells associated with headers?
  • Are they used for data and not presentation?
  • If tables are necessary for presentation, are they identified as such? E.g.: role=:"wairole:presentation"
• Are forms properly structured?
• Are forms organized by subject area?
• Are labels associated with input boxes?
• Are XHTML navigation landmarks taken from XHTML2? See: The XHMTL Role Attribute (Don't use 'skip to main content' - these roles are much more comprehensive.)
• Has a navigation section been identified on each page (role="navigation")?
• Do all navigation landmarks have a title including customer ARIA regions?
• Do Rich Internet Application widgets properly support ARIA and follow specified navigation rules?
• Do Rich Internet Application widgets follow the tab to container and arrow convention?
• Are keyboard mnemonic conventions consistent for context menus and a limited set of UI widgets?
• Are application and document areas in web pages properly defined to assist assistive technology with keyboard navigation? (A screen reader should not be in browse mode in a web application.)
• Does UI componentry provide rich hint, help, or tooltip text by establishing a relationship between the UI widget (could be a form control) and help information? For example, ARIA provides a describedBy and Tooltip relationship for these situations.
• Do all forms support extended ARIA form information, such as required and invalid?
• Are ARIA states and Properties synchonized with CSS style properties?
• Do UI components supporting multiple selections or options repeat option text?

Accessibility Review Protocols

The Fluid UX Accessibility Working Group has created a set of protocols for assessing accessibility. Code review is one aspect of an accessibility review.  If you'd like to do a deeper, more thorough evaluation, check out these additional resources.  These include inspecting markup as well as inspecting the behaviour of interfaces.

• Simple Accessibility Review Protocol:
  
  These are a simple set of heuristics for evaluating the general accessibility of a web application without need for complex assistive technologies or a lot of "under the covers" assessment of the HTML markup and so on.
• Comprehensive Accessibility Review Protocol for PC: This is a PC-based protocol for in-depth accessibility testing. It contains three accessibility test protocols for Fluid: Internet Explorer 7 with the AIS Toolbar Extension, the JAWS Screen Reader and the Web Accessibility Checker tool. A form for entering results and a completed form for reference are contained in attachments.
• Comprehensive Accessibility Review Protocol for Macintosh: This is a Macintosh version of the PC-based protocol.

Another useful guide for determining whether a site addresses accessibility through properly crafted HTML is the IBM Human Ability and Accessibility Center: Web Accessibility Checklist.