Added by Allison Bloodworth, last edited by Allison Bloodworth on Oct 24, 2007  (view change)

Labels:

Enter labels to add to this page:
Wait Image 
Looking for a label? Just start typing.

Sakai User Experience Walkthrough Report 

(Accessibility and usability heuristic evaluations combined with cognitive walkthroughs)

Evaluation Completed by:    Allison Bloodworth

URL:  http://sak-qa.berkeley.edu

Date: 7/24/07

Heuristic reviews and cognitive walkthroughs were performed on a MacBook Pro at 140 x 900 resolution (most common resolution) and Color LCD colors (most common novice resolution; if graphics work at this resolution, they'll work in higher resolutions) using Firefox 2.0.0.5 on a somewhat slow wireless connection.

See UX Walkthrough Heuristics for heuristics used in this evaluation.

Scope of Walkthrough

•    See UX Walkthrough Work Breakdown

User profile(s) and context of use:

Profiles will generally only be used for the cognitive walk-through, not the heuristic evaluation. However, users may be set up with a certain role in the system (e.g. student, instructor) for the heuristic evaluation.

•    See Sakai Personas

Scenarios

Scenarios will generally only be used for the cognitive walk-through, not the heuristic evaluation. However, it is fine to use these scenarios for the heuristic evaluation if it is helpful to evaluators in figuring out how to walk through the application. See Content Management Scenarios for a list of all possible scenarios. Evaluators will likely only cover some of the scenarios, in one of the roles, in their individual evaluations.

•    General Overview (heuristic evaluation)
•    Scenario 2
•    Scenario 3
•    Scenario 4
•    Scenario 5

Assumptions for this evaluation:

•    [List any additional assumptions used in evaluation]

 Positive Findings

All positive findings, from both the heuristic evaluation and the cognitive walk-through, should be placed in this section. 

Usability Positives Tool Evaluator
On the "Upload Files" page, the blinking "Processing" instruction gives users immediate and clear feedback that the system is working.
Resources AB
     
     
     
     
     
     
     
     
Accessibility Positives Tool Evaluator
     
     
     
     
     
     
     
     
     

Summary of Usability & Accessibility Issues Found 

All usability & accessibility issues found, in both the heuristic evaluation and the cognitive walk-through, should be placed in this section. 

Priority Legend:

High = Task cannot be completed

Medium = Task completed with significant effort and failed attempts

Low = Task completed with minor complications and/or annoyance 

Usability Issues Principle Link to screen shots Priority Suggestions for solution Tool Component Identified?
It is not clear to users how the different Resources tools in different sites relate to each other. Specifically, it is not clear whether Resources in My Workspace is its own separate instance of the tool, which doesn't include Resources from all the users' sites, or whether it is an aggregation of all the resources in the different Resources tools. The "Show/Hide Other Sites" link is styled strangely, making it hard to tell it is a link. Visibility of System Status, Consistency & Standards
  Medium Rename Worksite Setup's Resources tool to make it clear it is a global tool which shows the Resources for all the users' sites. Remove the underline from the arrow in front of the "Show/Hide Other Sites" link to make it clear that it is a standard link, and that the arrow (which is not part of the link) shows that there is something under it.
Resources Responsive Disclosure "Show/Hide" widget
When you just have one folder is not clear why there are two separate "Add" and "Actions" drop-downs. Especially since the "Actions" drop-down only has two options (Edit Details & Reorder) for a folder, is it really necessary to give the users extra information to parse?
Aesthetic & Minimalist Design, Recognition Rather than recall.
  Low
Consolidate the drop-downs as appropriate, or make them look clearly different (and perhaps locate them differently) to indicate their different functions.
Resources
 
"X" button on the "Upload Files" page allowing you to remove a file from the files to be uploaded is an unusual affordance, and not clearly associated with the file it affects. This is evidenced by the fact what it does must be explained in directions at the top of the page. It also doesn't work unless you have more than one file (e.g. it has no effect on a single file.
Consistency & Standards
  Medium-Low Reorganize the page and use visual design to better indicate the association between related components.
Resources  
The "Display Name" title on the "Upload Files" page for an uploaded file is not clear. Where will this name be displayed?
Match between the system and the real world.
  Medium-Low
Give instructions for entering this data and explain where it will be used/displayed, perhaps in pop-up help.
Resources (Pop-up) Help Component
The "Add Another file" button on the "Upload Files" page isn't clearly associated with the uploaded files.
Consistency & Standards
  Low
Re-organize the page to group related components and show relationships. Resources Add Another of the Same
When multiple files are uploaded, it isn't clear that the "Email Notification" is associated with all the uploaded files.
Consistency & Standards
  Low
Re-organize the page to group related components and show relationships. Resources
When a file is being uploaded on the "Upload Files" page, it seems that the "Cancel" button should work to cancel the action, but it doesn't. Even the "Stop" button on my browser didn't work to stop an overly long upload (over 10 minutes) and I had to log out. This was only an issue with a slow wireless connection.
Consistency & Standards, Help users recognize, diagnose & recover from errors
  Medium-Low Allow the Cancel button to work when files are in the process of being uploaded, or don't display it when it doesn't have an effect. Resources  
The directions on the "Upload Files" page are incorrect. They reference an "Upload" button, but the button is really called "Upload Files Now." It is an indication of a somewhat confusing interface that these directions have to be given at all.
Error Prevention, Consistency & Standards
  Medium-Low Re-organize the page to group related components. Put buttons which save changes in clear, standard places (consider having them always on the very bottom -- and maybe the top -- of the page, without directions underneath then as they are here).
Resources  
The "Access" label by itself is unclear. It is also not clear that you should be able to get to it from the "Edit Details" page. Inside "Edit Details" it is called "Availability & Access." More confusingly, there is a section on this page called "File Details," and an unlabeled section at the top. This makes it seem like the unlabeled section at the top and "Availability & Access" *are not* covered under the "Edit Details" page title.
Match between the system and the real world, Consistency & Standards
  Low
Consider a new label for Access on the main page, or pop-up help. Use the same label consistently in the tool. Consider a new organizational scheme for the information items in the Edit Details screen, and possibly re-naming it to better convey the different types of information it contains.
Resources (Pop-up) Help Component
In the "Optional Properties" section of "Edit Details," how does "Creator" and "Date Created" related to the uneditable "Created by" and "Created" fields at the top of the page? Where are these fields displayed? It seems that they are only displayed in this "Optional Properties" section and don't affect anything else, but this is not at all clear.
Match between the system and the real world, Consistency & Standards
  Medium
Remove redundant fields, or rename them to clearly show how they are different.
Resources  
In the "Optional Properties" section of "Edit Details," how should "Subject & Keywords" be entered? A comma separated list? Where are these fields displayed? Error prevention
  Medium-Low Give instructions for entering this data and explain where it will be used/displayed, perhaps in pop-up help.
Resources (Pop-up) Help Component
In the "Optional Properties" section of "Edit Details," how should the date-related fields be entered? Where are these fields displayed? Error prevention   Low Give instructions for entering this data and explain where it will be used/displayed, perhaps in pop-up help. Resources (Pop-up) Help Component
Users see different options when from the "Upload Files" screen they "Show Details for this item" vs. when they "Edit Details." How do you change the dates the files can be viewed in "Edit Details?" - bSpace bug ONLY
Consistency & Standards
  High Add missing options to "Edit Details."
Resources  
Used the Back button in Resources and got a blank screen. Wouldn't have known how to recover from this if I didn't have prior domain knowledge. After I go to the "Edit Details" screen, where I really just want to "View Details," I shouldn't have to feel like I'm canceling an action just because the page is misnamed. It would make more sense to be able to choose the back button.
Consistency & Standards
  High Users expect the back button to work based on their experiences with other web sites and web applications. Make it work. Perhaps also add Breadcrumbs to encourage users to navigate to higher level pages instead of using the back buttons.
Resources Breadcrumbs
On the "Edit Details" page, the user is allowed to "Change file type." This is almost never done in a traditional app; if the user wants to change the file extension, they access this by changing the name
Consistency & Standards
  Low Allow users to change the name & the file type at the same time
Resources
 
"Copy," "Move" & "Remove" links are presented in a completely not-standard format. It is very confusing as it is in essence a second somewhat hidden nav that appears below the global action links. These are normally treated as buttons in the rest of Sakai.
Consistency & Standards
  Medium If you need to operate on multiple items at once, in the current scheme for remove it is a column at the end with checkboxes. Then there is an update link on the bottom. I'm guessing Copy could have a single link on each line as announcements has an "Edit" link I tried to click a box an choose Copy but it didn't seem to do anything at all except turn the lines orange.
Resources  
I wasn't really sure what the color scheme in this tool was about--if it selects items, that  (and what to do next) should be made clearer.
Error Prevention, Consistency & Standards
  Medium
  Resources
 
Upload Download Multiple Resources link is very midleading for users looking for this function, as you can up/download multiple resources in the normal (Site Resources) screen and the only thing you can do at this link is use Web-DAV. It seems this name is a bit redundant, since the title above it is "Resources."
Consistency & Standards
  Low Come up with better naming conventions. E.g. "Up/Downlowad Multiple Resources with WEB-DAV." Resources should be called "Site Resources" on most sites; in My Workspace it should be called "All Resources." The main page could be called "Resource Viewer."
Resources  
Resources was in a strange state for me, with lots of the rows colored. I am not sure how it got that way, but may have had to do with copying or reordering files. I clicked on a clipboard icon to the left of the "Add" drop-down for the top folder, and the page revered back to it's normal color and apparently state. I have no idea what occurred there. Aha! It duplicated some files. I had no idea how to finish that duplication step, and clicking on that clipboard icon was not at all a logical way to complete that action. I guess the idea was that I was pasting copied items here, but that is not a logical workflow; it would seem from the context that the copy link would also create copies of the files.
Prevent Errors
  High   Resources  
I keep wanting to click the items in Resources to edit them instead of download them.
Prevent Errors
  Medium Perhaps we need separate links for download and other functions, and nothing should be linked from the resource title itself. I have to think about it almost every time to get it right when I'm editing.
Resources  
Somehow I created a strange red file which makes Resources blank when I click on it. It disappeared when I logged out and logged back in. - Can't reproduce. It appears this was the icon for a binary file, which was uploaded when I encountered an error somehow.
Prevent Errors
  High Don't use a red icon for this. Make it clear what has happened when there is an error instead of placing a red file icon there.
 Resources  
It is confusing that both the folder the user is in as well as the contents of the folder are shown. Though the containing folder is indented slightly, this isn't prominent and the user may not realize there is a distinction between the two. This is very different from the metaphors in most file systems where each folder in the hierarchy can be opened to view its contents, while maintaining the overall view of all the folders in the system. The metaphor to navigate to the containing folder for the current selection is different from the metaphor to navigate to its containing folder (there you use breadcrumbs), which can be confusing. Especially if the user navigates away from Resources and then comes back, when state is saved, it can be especially unclear where they are in the hierarchy.
Consistency & Standards, Recognition rather than Recall
  Medium Use a more standard presentation of the breadcrumbs which a user will recognize at a glance. It may be that the folder icons  and "Location: " header throw people off.
Resources Where am I/Breadcrumbs
When a URL is "uploaded", if the user enters http://, the // is changed to __ which breaks most links (though not Google, strangely enough). Users also cannot edit the URL after it's initially entered
Prevent Errors, Help users recognize, diagnose, and recover from errors, Consistency & Standards
  High   Resources
 


Accessibility Issue Principle Link to screen shot
Priority Suggestions for solution Tool Component identified?
             
             
             
             
             
             
             
             
             
             
             
             
             
             

Cognitive Walkthrough Worksheet

This section should be used by evaluators to keep track of the steps & screens they followed through the application in the cognitive walk-through. Any positive results or issues found should be included in the two sections above.

Scenario 1:  Brief description

# Step Screen Comments/issues Principle Suggestions for solution
           
           
           
           
           
           
           
           
           


Scenario 2:  Brief description

# Step Screen Comments/issues Principle Suggestions for solution
           
           
           
           
           
           
           
           
           


Scenario 3:  Brief description

# Step Screen Comments/issues Principle Suggestions for solution
           
           
           
           
           
           
           
           
           


Scenario 4:  Brief description

# Step Screen Comments/issues Principle Suggestions for solution