Sakai User Experience Walkthrough Report

(Accessibility and usability heuristic evaluations combined with cognitive walkthroughs)

Evaluation Completed by: [Insert evaluators name]

URL: http://bspace.berkeley.edu/portal/site/24cc0997-e57a-4f55-00ca-02eb0b35a6e7

Date: 22 Oct 07, 29 Oct 07

Heuristic reviews and cognitive walkthroughs were performed on a [Dell] at [screen resolution] resolution (most common resolution) and [color setting] colors (most common novice resolution; if graphics work at this resolution, they'll work in higher resolutions) using [Firefox and Win XP Pro SP2] at a [broadband] connection rate.

See UX Walkthrough Heuristics for heuristics used in this evaluation.

Scope of Walkthrough

• See UX Walkthrough Work Breakdown

User profile(s) and context of use:

Profiles will generally only be used for the cognitive walk-through, not the heuristic evaluation. However, users may be set up with a certain role in the system (e.g. student, instructor) for the heuristic evaluation.

• See Sakai Personas

Scenarios

Scenarios will generally only be used for the cognitive walk-through, not the heuristic evaluation. However, it is fine to use these scenarios for the heuristic evaluation if it is helpful to evaluators in figuring out how to walk through the application. See Content Management Scenariosfor a list of all possible scenarios. Evaluators will likely only cover some of the scenarios, in one of the roles, in their individual evaluations.

•    General Overview (heuristic evaluation)
•    Scenario 2
•    Scenario 3
•    Scenario 4
•    Scenario 5

Assumptions for this evaluation:

• [Performed based on an instructor]

Positive Findings

All positive findings, from both the heuristic evaluation and the cognitive walk-through, should be placed in this section.

Usability Positives	Tool	Evaluator
Clicking the help icon takes you directly to the help section for the tool it was called from.	all	JO
Sortable tables show an arrow icon beside a heading. This shows both the method of sort (ascending/descending) and the heading which for which the sort is based on.	all sortable tables (i.e. announcements and resources)	JO
Users are able to include only the tools they want for their site		JO






Accessibility Positives	Tool	Evaluator

Summary of Usability & Accessibility Issues Found

All usability & accessibility issues found, in both the heuristic evaluation and the cognitive walk-through, should be placed in this section.

Priority Legend:

High = Task cannot be completed

Medium = Task completed with significant effort and failed attempts

Low = Task completed with minor complications and/or annoyance

Usability Issues	Principle	Suggestions for solution	Tool	Component Identified?
General
Use of the term "Reset" for the mechanism to take the user back to the first page of the tool. Reset is often used to refer to clearing the information entered and not just returning to a previous location.	Consistency and standards	A term such as "Return" may be more appropriate	All
There are three methods to organize content. Method 1 uses up/down arrows seen in the Syllabus, Method 2 uses drag and drop seen in the Site Info, and Method 3 uses numbers in a combo box seen in the Forums	Consistency and standards	Choose one or at most two methods of organization.	Syllabus, Site Info, Forums	Syllabus Items, Tool Order, Organize Forums and Topics
Should the instructor be able to set permissions. Is this more of an admin function?	Aesthetic and minimalist design	Remove from the instructor's view	Announcements, Resources, Chat Room, Email Archive, Polls, Schedule, Assignments	Permissions
Both the Question Pools and Resources pages are used to manage a hierarchical system of information, but are designed with a different look and feel.	Consistency and standards	Choose one method to be used in all necessary areas	Resources, Quiz	Site Resources, Question Pools
The horizontal navigation bar used for navigation within a tool is different across the tools, some user lighter text to show selection, others use darker, some are have vertical bars to separate items, others do not, some have no underline for the selected item, and etc.	Consistency and standards	Choose one method to be used in all necessary areas	All
Syllabus
Complicated work flow if the instructor just wants to add a file as the Syllabus. (Syllabus > create/edit > add > add attachment > upload file	Flexibility and efficiency of use, Match between system and the real world	Add an option along side create/edit to add a file. (System should probably also be able to display the contents of the file and not just a link)	Syllabus
The Add Syllabus form and editor do not display the same as the actual output. The style of bullets changes from the editor to the output. Also the preview shows the Title in a different font size and colour than is actually outputed.	Consistency and standards	The preview should be identitcal in appearance to the output on the site. The user may also like to be able to control the style of the Title.	Syllabus	Add Syllabus
In the create/edit screen of the Syllabus tool, the "Preview" and "Reset" link perform the same actions.	Consistency and standards, Aesthetic and minimalistic design	If there are no other functions for the "Preview" link to that what is performed by the "Reset" link, "Preview" should be removed.	Syllabus	create/edit
Redirecting a syllabus prevents other posted syllabus from being displayed. This is the case even after adding a new syllabus and verifying it's output with preview.	Error prevention, visibility of system status	The system should inform the user that a redirected website will block all other syllabus, or display all of them sequentially.	Syllabus	Redirect
To remove a redirected syllabus (web site) the user has to erase the url and push the save button. For all other syllabus the user can remove after it by going to create/edit, which isn't even viewable if a redirect has been used.	User control and freedom, consistency and standards, error prevention	Provide an item in the syllabus list (create/edit) so that all of the syllabus can be managed in a like manor.	Syllabus	Redirect, create/edit
When re-ordering syllabus in the Syllabus Item list, the user does not need to click the update button for the changes to take effect. Update is only needed for removal.	Error prevention, User control and freedom, Visibility of system status	Require that the user perform an update before the reordering takes effect, or rename the button to "Remove". Also disablling the button until an update/removal is needed will signal to the user when it is needed.	Syllabus	Syllabus
Home
On the "Home" page, clicking "Options" under Worksite Information allows you to add text. However, clicking on "Options" under Recent Announcements provides no ability to add text.	Consistency and standards	Provide a link to the announcements editing page.	Home	Recent Announcements
The "Title" and "URL" text fields are both smaller in width than the "Description" text area	Consistency and standards	The text fields do not limit the input to their size, so they should be equal in width to the Description text area, allowing for maximum visibility of information.	Home	Worksite Information > Options
Site Info
Should an instructor be able to add/remove users and change their role? Is this more of an admin function?	Aesthetic and minimalist design	Remove these features from the instructor, or put them under a subheading	Site Info
The Edit Tool and Tool Order links have the same function but different implementations of it (with the exception of the ordering option for tool order)	Consistency and standards, Aesthetic and minimalist design	They should probably be combined or have the titles changed so that one indicates for keyboard use and the other for mouse.	Site Info	Edit Tools, Tool Order
Slow to add/remove tools	Visibility of system status	There is a message displayed after the tool has been added/removed. There should also be a message indicating that the process is in progress.	Site Info	Tool Order
The user can drag to add tools but can't drag to remove tools	Consistency and standards	The user should be able to drag the tool off of the column and drop it either onto the available tools column or drop anywhere off the current column to remove.	Site Info	Tool Order
Removed tools do not show up in available tools until after a save is performed.	Visibility of system status	Tools should show up in the available tools list as soon as they are removed from the list of current tools.	Site Info	Tool Order
The "Reset" button is used to put the order of the tools to the default ordering. The term "Reset" is also used at the top as a means to return the tool to it's start page.	Consistency and standards	One or both of these items should be renamed. The "Reset" button could be renamed to "Default Order".	Site Info	Tool Order
Assignments
The descriptive text is unclear as to whether is is describing the combo boxes above or below it.	Error prevention	Information in the Assignment form needs to be borken up by white space to better seperate the sections.	Assignments	Add
Polls
When adding a poll, the user has to add the question and all possible answers on different pages. The only item that persists across pages is the question. If the user forgets what answers were already entered, they have to save, leaven and exit.	Recognition rather than recall	The user should be able to add the question and all possible answers on the same page.	Polls	Add
Schedule
When setting the time for an event, the time zone is displayed after the time combo boxes. However, the user is unable to change the time zone.	Aesthetic and minimalist design	The time zone seems to be set by the server, if the user has no control over it, it should not be displayed here.	Schedule	New Event
The legend below the Calendar is quite large and contains many items that may never appear on any one calendar all at a time.	Aesthetic and minimalist design	It would be better to only include those items which are currently visible in the calendar. Another solution would be to have a tooltip for the icon rather than include the icon in the event's title.	Schedule	Calendar
To scroll through the hours on the calendar, the user has to click on the links for earlier and later.	Consistency and standards	Users are often more familiar with using a scrollbar.	Schedule	Calendar
Announcements
The update button in the Announcements section is only used for removing announcements. This can cause confusion as the user can sort the list based on the table headings; however these changes don't affect the announcements posted on the site.	Error prevention, User control and freedom, Visibility of system status	Rename the button to "Remove". Also disabling the button until an announcement has been checked for removal. Another option would be to have a removal button next to each announcement instead of a check box.	Announcements
Resources
It is not clear what the resources are for. Items such as Quiz and Syllabus allow the user to upload the resource directly and don't seem to link to the resources for the user to select an item from there.	Consistency and standards, Aesthetic and minimalist design, Match between system and the real world		Resources
When viewing other site's resources the user is given the same access level to the files but does not have access to the sort methods that they would on resources from their own site. Also, the items are listed with all of the information that the items in the user's table but the headings aren't easily matched.	Consistency and standards	When selecting a folder from another site it should be broken off to show the table headings for the columns that are displayed. This will allow the user to sort, understand what the information is, and separate it from the other resource folders available.	Resources	Site Resources
When creating a new folder, uploading a file, or adding a web link there is an ' X ' at the far right. The tooltip for the ' X ' says "Remove this item". Clicking the ' X ' does nothing and there is already a cancel button at the bottom.	Consistency and standards, Aesthetic and minimalist design	remove the ' X '	Resources	Upload files, Create folders, Add web links (urls)
When adding a new folder there is an option to add another folder. This can be done when the first folder isn't created. Also it is unclear whether this other folder is meant to be a sub folder to the original or the one you are creating.	Error prevention	Only show the add another folder link once the original folder has been specified, or have an add multiple folders option. Also allow the user to add subfolders for the newly created folder, thereby distinguishing the levels of the folders created.	Resources	Create Folders
The text document and the HTML page result in similar documents. The exception being that the HTML page has more formating options	Consistency and standards	Rename the html page to something like rich text document and/or remove the simple text document.	Resources	Create HTML page, Create text document
The options section says " Select or unselect resources types to indicate whether resources of that type can be created in this site." Currently it is only a list of checked items which cannot be edited, except for Form item. Unchecking Form item and updating does not seem to have any effect.	Consistency and standards, Aesthetic and minimalist design	If these items are set by another user (i.e. an admin) this section shouldn't be called options, as all other instances of options contain editable options. It may be better to include it as a list somewhere on the main resources page or to omit it completely; as the user can easily determine the types of resources by hovering over add.	Resources	Options
When hovering over add or actions when they are close to the bottom of the browser window, the context sensitive menus are cut off.	Visibility of system status, Error prevention	The menus should open upwards when at the bottom of the screen instead of always opening downwards.	Resources	Site Resources
Can't collapse items into the top level folder.	Consistency and standards, Aesthetic and minimalist design	Allow the user to collapse contents of the top level folder.	Resources, Drop Box
Roster
There are two views of the roster, Overview and Pictures. The information provided for the picture view is cut off.	Aesthetic and minimalist design, Recognition rather than recall	Rather than having two separate pages, there should be one page with an option (e.g. a check box) to show pictures. When show pictures is selected the image should appear in a new Pictures column next to the name.	Roster
The term "Roster" may not be immediately relevant to all users	Consistency and standards, Match between system and the real world	A term such as course list, course users, or course participants may be more applicable.	Roster
Chat Room
Chat room does not show current user in the "users in chat" list, to the current user. This prevents the user from knowing if others can see they are there.	Visibility of system status	The users should see everyone available in the chat, even themselves	Chat Room
Email Archive
There are two separate tools, one for sending e-mail (Mailtool) and one for receiving e-mail (Email Archive).	Match between system and the real world, Consistency and standards, Aesthetic and minimalist design	The MailTool and Email Archive should be rolled into one Email tool. If the instructor is the only one who needs to send e-mails this portion should be blocked from the students. Users are used to having their email clients able to both send and receive e-mail.	Email Archive, Mailtool
Forums
When adding a new Topic the user selects "New Topic", when the page opens it is called "Topic Settings"	Consistency and standards	Keep the name the same, the page should also be called New Topic	Forums	Topic Settings
The buttons at the bottom of the Topic Settings page include "Save Settings" and "Save Settings & Add Topic". The first implies that only the settings are being saved and the second that the settings and the topic are being saved. In actuality "Save Settings" saves the topic and the other saves and opens a new page to add another topic.	Match between system and the real world, Consistency and standards	Change the names to something like "Save Topic" and "Save & Add Another Topic"	Forums	Topic Settings
Gradebook
When a category at the top is selected the text colour becomes a lighter shade to indicate that it is the currently selected category. However, the Upload/Import link is always a lighter colour	Visibility of system status, Consistency and standards	Make the text colour for Upload/Import follow the same conventions of the other links, being a darker shade when not selected.	Gradebook
News
Descriptive text between Page Title and URL text fields is not clear as to whether it refers to the field above or below.	Error prevention	Use whitespace to separate it from the lower field.	News	News Options
Not clear that the URL field requires the URL for an RSS feed	Error prevention	Indicate that an RSS feed is required. Change "URL" to "RSS feed URL"	News	News Options
Web Content
The web content does not show the browsers icons when opened in a new page. Many users may not know the keyboard shortcuts for those actions.	Recognition rather than recall	Allow the page to load with a normal browser view, or provided icons within the browser to fulfill those functions.	Web Content
Wiki
When editting a wiki, clicking on the modifier icons for bold, italic, and etc, repeatedly doesn't remove a modification but adds it again instead.	Consistency and standards, Error prevention	A second click should remove the modification	Wiki	Wiki Editor
Home and View appear to perform the same function. Both bring the user back to the home page of the wiki.	Aesthetic and minimalist design, Consistency and standards	If it performs the same function remove "view"	Wiki
Mailtool
There are two separate tools, one for sending e-mail (Mailtool) and one for receiving e-mail (Email Archive).	Match between system and the real world, Consistency and standards, Aesthetic and minimalist design	The MailTool and Email Archive should be rolled into one Email tool. If the instructor is the only one who needs to send e-mails this portion should be blocked from the students. Users are used to having thier email clients able to both send and recieve e-mail.	Email Archive, Mailtool
Drop Box
Can't collapse items into the top level folder.	Consistency and standards, Aesthetic and minimalist design	Allow the user to collapse contents of the top level folder.	Resources, Drop Box

Accessibility Issue	Suggestions for solution	Tool
General
It isn't always clear which item is selected when tabbing through a page. The selection bounding box is not always visible.
After selection of a tool from the left navigation bar, the tab traversal resets and cycles back from the start.	It would be more helpful to the user if the tab traversal starts inside of the tool's frame	all

Fluid Components

Community

Other Fluid Projects

More Information

Page Operations

Browse Space