|
The Fluid team has selected three inspection methods to evaluate current and prospective products: Heuristic Evaluation, Cognitive Walkthrough, and under-the-covers Code Review. In addition, a set of questions which may be employed with any of the three methods is included in its own section: Additional Questions for All Reviews.
Each of the methods may be used to address usability, accessibility, or both. The only difference is the set of heuristic principles applied or questions asked during the review. Checklists for both usability and accessibility are offered here for each method.
It is not necessary for you to use all three methods to contribute to the Fluid UX walkthrough endeavour. Nor must you address both accessibility and usability. If you do participate, please use the following checklists to guide your explorations, and use the appropriate UX Walkthrough Report Template to report your findings.
I. Heuristic Evaluation
An heuristic evaluation is an expert review of a web page with reference to explicit principles termed "heuristics". During the evaluation, reviewers see how well a page fulfills the basic requirements for usability or accessibility.
To conduct an heuristic evaluation of a service offered through the web, travel through the pages of the site, reflecting on each of the listed principles, and recording compliance and violations. The heuristic principles should also be kept in mind during cognitive walkthroughs.
Heuristic Principles for Usability
This section lists the original Nielsen and Molich heuristics , as refined by Nielsen.
Visibility of system status
The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.
Match between system and the real world
The system should speak the user's language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.
User control and freedom
Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.
Consistency and standards
Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.
Error prevention
Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.
Recognition rather than recall
Minimize the user's memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.
Flexibility and efficiency of use
Accelerators - unseen by the novice user - may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.
Aesthetic and minimalist design
Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.
Help users recognize, diagnose, and recover from errors
Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.
Help and documentation
Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.
More detail on each of the principles can be found in be found in an expanded list from Deniese Pierotti of Xerox, which itemizes specific things to look for when evaluating a system with Nielsen's Heuristics. See Heuristic Evaluation - A System Checklist.
Heuristic Principles for Accessibility
The Fluid UX Accessibility Working Group has created a set of protocols for assessing accessibility:
- Simple Accessibility Walkthrough Protocol: This is a set of simple heuristics for evaluating the general accessibility of a web application without need for complex assistive technologies. It provides a simple technique that anyone can learn while doing UX Walkthroughs.
A paper from Claire Paddison and Paul Englefield provides a list of nine heuristic principles for accessibility evaluations:
Paddison and Englefield include in their paper a general discussion of the heuristic approach. This is recommended reading for all reviewers.
II. Cognitive Walkthrough
A cognitive walkthrough is a step-by-step exploration of a page to see how well a particular type of user - represented by a persona - will be able to accomplish his or her objectives.
The following must be set down explicitly before the the walkthrough process is begun:
- a user goal - the specific result desired by the user and motivating the interaction
- a persona which:
- is adequate to judge what knowledge the user may plausibly be expected to have
- specifies the particular needs, preferences, and limitations the user may have
Note that separate walkthroughs may be needed for each persona, although some issues will likely show up in more than one walkthrough, resulting in later walkthroughs going more quickly than earlier ones.
Addressing Usability in a Cognitive Walkthrough
Work out the sequence of steps the user should go through, to accomplish the goal.
For each step, ask the following questions:
- Will the user know what to do at this step? Is complex problem solving needed to figure out what to do?
- If the user does the right thing, will they know that they did the right thing, and are making progress towards their goal? Is complex problem solving needed to interpret the feedback?
Addressing Accessibility in a Cognitive Walkthrough
An accessibility walkthrough follows procedure similar to the cognitive walkthrough described above, with the difference that the user has one of a number of disabilities such as low vision, blindness, impaired hearing, motor control limitations, or cognitive issues.
In an accessibility walkthrough, the main consideration is how these limitations affect the use of websites or software. For example: blind persons and some persons with limited motor control need keyboard-only operation; some persons with cognitive issues need visuals that reinforce text; persons with low vision must enlarge page content; deaf people require video captioning and visual, rather than auditory, prompts. You also must consider the assistive technology the user will use. Examples include screen magnifiers such as ZoomText, screen readers such as JAWS, or combination screen readers/enlargers such as Kurzweil 3000.
Before proceeding with the formal walkthrough, it is useful to perform the following steps:
- Assess the overall layout and structure of each page.
- Play around with the layout: enlarge the font size; change the size of the window (bigger and smaller); adjust the resolution.
- Use the Tab key to traverse the entire page.
To perform an accessibility walkthrough, adopt a persona, select a goal, and perform the steps necessary to accomplish the goal. At each step, ask the following questions:
1. Will the user know what to do at this step?
- Is complex problem solving needed to figure out what to do?
- Are the cues provided accessible to the user?
- Can the cues be understood by someone who doesn't process text well?
- Can the cues be found by someone who can't scan the screen easily?
2. Will the user be able to carry out the required action?
- Can it be performed easily by a keyboard-only persona?
- Can it be performed without visual monitoring?
3. If the user does the right thing, will they know that they did the right thing, and are making progress towards their goal?
- Is complex problem solving needed to interpret the feedback?
- Is the feedback accessible to the user, and can they find it (as for cues)?
To conduct an actual accessibility walkthrough and assessment, it is recommended that the reviewer select a persona to adopt and then follow one the detailed protocols listed in the section on heuristics (above):
III. Code Review - Examining the HTML Under the Covers
Websites have the opportunity to provide additional information to persons with disabilities to help them understand and navigate more easily. Some of this information is visible to all website users and can enhance their experience. Some of it is transparent to sighted users and provides context for persons with visual, auditory or motor limitations.
The Fluid UX Accessibility Working Group has created a set of protocols for assessing accessibility. These involve the use of tools for a somewhat automated approach.
Code reviews can also be accomplished through direct examination of HTML. A useful guide for determining if a site addresses accessibility through properly crafted HTML is IBM's Web Accessibility Checklist:
Under-the-covers Questions
- Is content separated from presentation through CSS?
- Are buttons and other controls text-based?
- Are tables used for data and not layout?
- Does page content linearize?
- Do pages have frames?
- Are they necessary? If so, do they have meaningful titles?
- Are headings hierarchical?
- Is punctuation provided in headings to control screen readers?
- Are tables properly structured?
- Do they contain headers and summaries?
- Are table cells associated with headers?
- Are they used for data and not presentation?
- If tables are necessary for presentation, are they identified as such? E.g.: role=:"wairole:presentation"
- Are forms properly structured?
- Are forms organized by subject area?
- Are labels associated with input boxes?
- Are XHTML navigation landmarks taken from XHTML2? See: The XHMTL Role Attribute (Don't use 'skip to main content' - these roles are much more comprehensive.)
- Has a navigation section been identified on each page (role="navigation")?
- Do all navigation landmarks have a title including customer ARIA regions?
- Do Rich Internet Application widgets properly support ARIA and follow specified navigation rules?
- Do Rich Internet Application widgets follow the tab to container and arrow convention?
- Are keyboard mnemonic conventions consistent for context menus and a limited set of UI widgets?
- Are application and document areas in web pages properly defined to assist assistive technology with keyboard navigation? (A screen reader should not be in browse mode in a web application)
- Does UI Componentry provide rich hint, help, or tooltip text by establishing a relationship between the UI widget (could be a form control) and help information? (E.g. ARIA provides a describedBy and Tooltip relationship for these situations)
- Do all forms support extended ARIA form information (such as: required and invalid)?
- Are ARIA states and Properties synchonized with CSS style properties?
- Do UI components supporting multiple selections or options repeat option text?
IV. Additional Questions for All Reviews
The questions in this section may be helpful in the evaluation of the overall usability of an interface. While they do not represent a checklist which should be used to report usability or accessibility problems, they address things evaluators should be looking for during any review. Items pertaining specifically to accessibility are flagged with [a11y]. (The string "a11y" is a common abbreviation for "accessibility".)
Text
- Is plain language used?
- Is vocabulary familiar to the intended user, avoiding system-oriented terms?
- Does a page's title accurately describe its purpose?
- Are titles and headings clear and distinct?
- Do headings precede related paragraphs?
- Do headings make sense out of context?
- Are related sub-points or sub-navigation links grouped into lists?
- Are sentences and paragraphs short and to the point?
- Have links been front-loaded into sentences?
- Do links make sense out of context?
- Are links unique?
- Are separate ideas kept in separate sentences and paragraphs?
- Is the use of all-upper-case text limited to acronyms?
- Is additional context provided by a mouseover tip when needed (e.g. for links)?
- Are words or sentences emphasized by both colour and weight?
Non-text media
- [a11y] Is there a text alternative that provides the same information for non-text material?
- [a11y] Is there a synchronized alternative for multimedia?
- [a11y] Can users control speed of presentation?
- Is there adequate contrast within images?
Layout and presentation
- Is the order of information natural and logical?
- Is similar information presented in similar, consistent ways?
- Are bulleted or numbered lists used?
- Is white space used to separate ideas?
- Is unnecessary material displayed (clutter)?
- Is flashing that could cause seizures used?
- Is there distracting blinking, flashing, or animation?
- Can users suppress distracting effects?
- Can foreground elements - either text or images - be easily distinguished from the background?
- [a11y] Can browser settings control text size, image size, and colours?
- [a11y] Does the layout adjust to font size and image size?
- [a11y] Can the shape of the layout be adjusted?
- [a11y] Does the layout retain its integrity at 400% magnification?
- [a11y] Is important information emphasized in non-colour ways?
- Are illustrations provided for important concepts?
- Are there summaries of pages and sections of content?
- Are menu items numbered to aid navigation?
- Will vocabulary be familiar to users and not system-oriented?
- Are platform conventions followed?
- [a11y] Can all controls be operated via the keyboard?
- [a11y] Is keyboard-based navigation properly coordinated with screen reader keystroke assignments?
- [a11y] Are there controls that require rapid or time-limited responses?
- [a11y] Are users provided notification of time-out and allowed to request more time?
- [a11y] Does UI Componentry provide rich hint, help, or tooltip text?
- [a11y] Is this accomplished by establishing relationships between the UI widget (could be a form control) and additional help information? (eg. ARIA provides a describedBy and Tooltip relationship for these situations)
Navigation and control
- Does the system always keep users informed about what is going on, through appropriate feedback, within reasonable time? (Visibility of system status)
- Can users easily recover from errors, unintended actions, or actions that did not lead to desired results (eg undo, back)?
- Is confirmation required when an action is difficult or impossible to undo?
- Is navigation consistent and intuitive?
- Do links match destination titles?
- Does the application follow platform and usability conventions?
- Are well-known pictures and symbols used? (eg. "?" for help, "<" for back, "i" for information).
- Are similar operations or tasks performed in similar ways?
- Are there error-prone operations that can be eliminated?
- Does the application support user memory?
- Are accelerators provided for experienced users?
- Can repetitive actions or frequent activities be made easier?
- Are error messages easy to understand and respond to?
- Is help information and/or documentation available?
- Is it easy to search?
- Is information focused on the user's task?
- Does it provide concrete steps?
- Are illustrations provided to make instructions easier to understand?
- Are there features (eg a site map, navigation bar) that help users find content and navigate?
- Are there topic maps for complex content?
- Is there a home page link?
|
Page Operations
Browse Space
Posted by
I've attached a protocol for testing with JAWS that we use at the UAC. I tried pasting it here, but the text got all messed up.
Mike