Skip to end of metadata
Go to start of metadata

(Accessibility and usability heuristic evaluations combined with cognitive walkthroughs)

Evaluation Completed by: Allison Bloodworth

URL:  http://my.ubc.ca....

Date: 9/13/07

Heuristic reviews and cognitive walkthroughs were performed on a MacBook Pro at 1600x 1200 resolution (most common resolution) and Dell2001FP colors (most common novice resolution; if graphics work at this resolution, they'll work in higher resolutions) using OS 10.4.10 at a T-1 connection rate.

See UX Walkthrough Heuristics for heuristics used in this evaluation.

Scope of Walkthrough

•    See uPortal Evaluation Plan

User profile(s) and context of use:

Profiles will generally only be used for the cognitive walk-through, not the heuristic evaluation. However, users may be set up with a certain role in the system (e.g. student, instructor) for the heuristic evaluation.

•    See Sakai Personas

Scenarios

Scenarios will generally only be used for the cognitive walk-through, not the heuristic evaluation. However, it is fine to use these scenarios for the heuristic evaluation if it is helpful to evaluators in figuring out how to walk through the application. See uPortal Content Management Scenariosfor a list of all possible scenarios. Evaluators will likely only cover some of the scenarios, in one of the roles, in their individual evaluations.

•    General Overview (heuristic evaluation)
•    Finding.....
•    Scenario 3
•    Scenario 4
•    Scenario 5

Assumptions for this evaluation:

•    [List any additional assumptions used in evaluation]

 Positive Findings

All positive findings, from both the heuristic evaluation and the cognitive walk-through, should be placed in this section. 

Usability Positives

Tool

Evaluator

Nice affordance on the ^ which indicates a portlet can be expanded or contracted

portlet container

AB

In the NewsReader, nice icons and pop-up messages for 'Add Feed,' 'Add Folder,' etc.

NewsReader

AB 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Accessibility Positives

Tool

Evaluator

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Summary of Usability & Accessibility Issues Found 

All usability & accessibility issues found, in both the heuristic evaluation and the cognitive walk-through, should be placed in this section. 

Priority Legend:

High = Task cannot be completed

Medium = Task completed with significant effort and failed attempts

Low = Task completed with minor complications and/or annoyance 

Usability Issues

Principle

Link to screen shots

Priority

Suggestions for solution

Tool

Component Identified?

When session times out, portal changes to two tabs, which is disconcerting, and doesn't give the user a message about what has happened

Visibility of System Status

 

Low

Give user a message about what has happened (e.g. the session timed out and they need to log in again).

portal

Error Message Container

When the login fails, it is difficult to see the error message because it is styled in exactly the same manner as the field labels. It is also strange that the error message is both at the top and bottom of the "Login" box.

Visibility of System Status

 

Low

Create a container for the error message which will be used as a standard across the portal/portlets

portal

Error Message Container

When you expand or contract a portlet using the ^ icon (or it seems, possibly take any action on a portlet), the page shifts downward so that this portlet is at the very top of the page, which is disconcerting to the user.

Consistency & Standards

 

Low

Keep the page where it is.

portal

 

When a user expands a portlet using the 'window' icon, they are 'taken out' (at least that's what they are told) of the portal without warning, and may not understand where they are. The 'Return to portal' link, which takes them back, is hard to see and uses terminology they may not understand.

Consistency & Standards, Match between system and the real world

 

Medium

Use clearer text to explain what happens in the pop up which comes up when a user mouses over the 'window' icon. Use text and visual design in the wrapper for the expanded portlet to make it clearer the user is still in the portal. Make the 'Return to portal' link much more prominent, and perhaps explain more about what that means. It looks graphically like you are sort of in the portal, so its difficult to understand exactly what that means. And is that distinction even helpful to the user?

portal

Container for expanded portlets

When the user expands a portlet, the container on the upper left groups what appears to be the title of the portlet, e.g. "Student Services Center" and the 'Return to portal' link together in a way that almost makes them look like breadcrumbs and is not at all prominent. There is also an issue with the graphic design where the words are not centered vertically on the background.

Consistency & Standards

 

Medium

Move the portlet title to the center of the screen, and disconnect it from the 'Return to Portal' link. This should also help to make the 'Return to portal' link more prominent.

portal

 

In certain portlets (e.g. Student Services Center"),  users have to perform another click to get to the actual student services center.

Flexibility & Efficiency of Use

 

Low

Put more than one link in a portlet window. Save valuable portlet real estate by aggregating these links. A single link should not have its own window.

portlets

 

When the user clicks the "X" button they get a message asking , 'Are you sure you want to remove this channel,' with no explanation of what that means, or how to reinstate it if the user changes their mind.

User Control & Freedom

 

Medium

Explain this in the confirmation message.

portal

Did you really want to do that? (w/optional help)

When the user clicks on certain links (e.g. news article links in the News Reader, the Help icon), they are taken, without warning, to a new window. This could be exceptionally disconcerting to users of screen readers.

Consistency & Standards, User Control & Freedom

 

Medium-Low

Give an indicator when a link will open a new page.

portal

 

If the users are in the middle of one action, e.g. editing the NewsReader, and go to another tab, when they come back to the NewsReader tab the action has not been canceled and there is no visual indication the user is in a different 'mode.'

Consistency & Standards, Recognition Rather Than Recall

 

Medium-Low

Indicate that the portlet is in an edit state visually and with mark-up that will make this obvious to a screen reader. Consider asking whether the user wants to cancel an in-progress action if they navigate to another tab.

portal

 

When editing the newsreader, it is very difficult to see the instructions that say 'Click on Ok to confirm your changes..."

Consistency & Standards

 

Low

The visual treatment (e.g. color, shape, alignment) of all the buttons (and even the text) on this page is too similar. There should not be any need for instructions like this if the page visual design and flow (for screen readers) made it clear that it was necessary for the user to take an action.

NewsReader

 

In the NewsReader "My Feeds Subscriptions" folder it looks like the icons (e.g. the folders and RSS icons) should be clickable. However, only the the labels are clickable for the icons which display by default. This is even more inconsistent because you activate other icons (those which allow you to perform actions on the current folder or feed) by clicking on them.

Consistency & Standards

 

Medium

Make the folder & feed icons clickable.

NewsReader

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 


Accessibility Issue

Principle

Link to screen shot

Priority

Suggestions for solution

Tool

Component identified?

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Cognitive Walkthrough Worksheet

This section should be used by evaluators to keep track of the steps & screens they followed through the application in the cognitive walk-through. Any positive results or issues found should be included in the two sections above.

Scenario 1:  Brief description

#

Step

Screen

Comments/issues

Principle

Suggestions for solution

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 


Scenario 2:  Brief description

#

Step

Screen

Comments/issues

Principle

Suggestions for solution

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 


Scenario 3:  Brief description

#

Step

Screen

Comments/issues

Principle

Suggestions for solution

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 


Scenario 4:  Brief description

#

Step

Screen

Comments/issues

Principle

Suggestions for solution

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 


  • No labels