User Experience Walkthrough Report 

Customized Walkthrough Reports

Available for:

• Sakai 
• uPortal 

 Generic Walkthrough Report

(Accessibility and usability heuristic evaluations combined with cognitive walkthroughs)

Evaluation Completed by:    [Insert evaluators name]

URL: https://my.ubc.ca/render.userLayoutRootNode.uP

Date: September 13, 2007

Heuristic reviews and cognitive walkthroughs were performed on a [computer type] at [screen resolution] resolution (most common resolution) and [color setting] colors (most common novice resolution; if graphics work at this resolution, they'll work in higher resolutions) using [Browser and Operating System] at a [type of connection is internet required] connection rate.

See Heuristic Evaluation for heuristics used in this evaluation.

Scope of Walkthrough

•    Managing Conent

User profile(s) and context of use:

Profiles will generally only be used for the cognitive walk-through, not the heuristic evaluation. However, users may be set up with a certain role in the system (e.g. student, instructor) for the heuristic evaluation.

•    Ed McClellan (student)

Scenarios

Scenarios will generally only be used for the cognitive walk-through, not the heuristic evaluation. However, it is fine to use these scenarios for the heuristic evaluation if it is helpful to evaluators in figuring out how to walk through the application.

• General Overview
• Ed has just completed his first-time student registration process and is now going to the portal to check his institution email for the first time. Ed is also curious about what online services are offered and what the university portal is like. (portal, login, webmail)
  Kathy Moore (done)
• Ed is finishing registering for spring semester classes for his freshman year and needs to clarify and verify the registration deadlines. (calendar and/or announcements)
  Kathy Moore (done)
• Ed wants to complete registration for spring semester classes for his freshman year. (SIS, course browsing, course registration, e-commerce)
• Ed wants to find and purchase used textbooks for his classes (LMS, web content, e-commerce)
• Ed needs to find the location of his class on the campus map.
• Ed needs to check his course for current assignment details. (LMS)
• Ed is searching for his instructor's contact information to email a question regarding the assignment (LMS, faculty directory)
• Ed wants to IM/chat with his classmates regarding the assignment (LMS, chat)
• Ed needs to submit his assignment. (LMS, dropbox)
• Ed wants to check his grade for his assignement (LMS, SIS)
• Ed needs to see when the next bus/shuttle will be coming and if he has enough time to finish watching his TV program before heading to the bus stop.
• Ed missed the college football game last weekend and wants to check the scores, coverage, news, and/or highlights from the game. (news, athletics content)
• Ed desires to find out about community/cultural events for the upcoming weekend when his family is coming to visit. (calendar, message boards)
• Ed would like to find out if anyone is selling a cheap couch that he can put in his dorm room. (classifieds)
  Kathy Moore (done)
• Ed wants to know when his library books are due - alternatively, he wants to see if the books he wants are available for check out. (library)
• Ed needs to pay for parking. (e-commerce)
• Ed wants to know what is on the menu for dinner at the cafeteria. (web content)
• Ed would like to find out the hours the recreation center is open and what time the box-aerobics class is scheduled. (web content)
• Ed is curious to see what part-time employment opportunities are available (web content, career services)
• Ed has finally acquired a set of wheels and wants to know what the availability and costs are for parking on campus.

Assumptions for this evaluation:

•    [List any additional assumptions used in evaluation]

 Positive Findings

All positive findings, from both the heuristic evaluation and the cognitive walk-through, should be placed in this section. 

Summary of Usability & Accessibility Issues Found 

All usability & accessibility issues found, in both the heuristic evaluation and the cognitive walk-through, should be placed in this section. 

Priority Legend:

High = Task cannot be completed

Medium = Task completed with significant effort and failed attempts

Low = Task completed with minor complications and/or annoyance 

Usability Issues	Principle	Link to screen shots	Priority	Suggestions for solution	Tool	Component Identified?
General
Can I customize my pages:  "main", "webmail", "info finder", e-learning", etc.?  If so, I can't figure out how to do anything except make portlets go away.	User control & freedom		High			Page customizer
User shoiuld be able to organize the portlets on a page to fit their needs.	User control & freedom		High	Utilize the "organizer" component.  It will need to be enhanced for this particular context		Layout manager (organizer)
Prior to Login
Main
Bold text in the "Student Service Centre"  draws the eye yet should be unessasary.  The link itself should look like a link and be descriptive enough not to have text above it explaining to click it to launch the app.	Minimalist Design - Every unit of information on the page competes with relevant units of information.		Low
Clicking on the open close carrot to close the portlet, in this case "Student Service Centre" leaves the screen in a strange place.  The screen auto-scrolls so the carrot is at the top of the page (see screen shot).  The screen movement is jarring and there is no need to bring the carrot into focus in this case.	Jarring & user looses sense of where they are since screen changes automatically		Low			Minimizer
After maximizing the portlet, it is difficult to tell how to get back to the view I was just on.  User has to see and click on the Blue "Return to Portal" link near the top left of the screen.  Because of the location and the background it is very easy to miss.	User control & freedom - users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state quickly.		Medium	Make the link pop by making it look more like a link and move it away from the logo, etc.		Undo and/or Back & "Open new window" notifie
When a link will take the user to a new a window there should be some indication of that on the page for both sighted and non-sighted users.  Window management is complex for some users and they will have difficulty getting back to the original window...especially if they were expecting the object to open within the existing window.  This is especially true in the weather portlet where I expect clicking on "click for forecast" to show me more information in-line as it does when I'm on the weather website, weather.com.  Even if the user knows they are opening an application, we should not assume they know it will open in a new window.	Visibility of system status & User control & freedom - users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state quickly & Consistency & Standards (with other apps)		Medium	Add text or an icon to indicate a new window will open.  If possible make it  easy to get back to the original from the new window	All portlets that spawn a new window	Undo and/or Back
After logging in, I have both the "student service centre" and the "management system portal" portlets available to me.  The system should be smart enough to know who I am and tailor the portlets to me.  I assume I would not have access to both of the systems.	Minimilist Design - screens should not contain information which is irrelevant or rarely needed.		Low	Only show relevant portlets based on who the user is.		Who am I?
What if I want to change the cities showing in the weather portlet?  As a student, I very likely want to see the weather back home along with in Vancouver (in the case of UBC).  I can't figure out how to change it.	Flexibility and efficiency of use		High	Allow users to customize the cities.  If this is already the case, make it clear to users from this page how to do it.
Choosing the X in the upper right hand corner of a portlet displays a message that says "are you sure you want to remove this channel?"  No, I'm not, what does that mean to remove it.  Can I ever get it back.  Browsers use this icon to close (rather than remove) the window.  After I removed it, it seems to be gone forever.	Consistency and standards & User control & freedom - support undo and redo.		High (because you can't get the portlet back once it's gone)			Portlet manager (remove & re-add, etc.) Window closer
Webmail
Webmail settings form's layout is very difficult to read:  -inputs are not visually connected to their label because of so much space between them -some text is bolded which makes it pop but it doesn't seem necessary ("ascending" & "descending" for instance) -"When sending messages..." message's control should be layed out so they read across ("Place copy in" should be on same line and in front of the radio buttons. -The input controls are all squished together making it difficult to tell what goes with what -There is a "YesNo" in the top left hand side of content area.  I have no idea what it is. -"Display full message headers" control should be radio buttons since it's only 2 choices.  It is always better to see all the choices and the radio buttions can be places in 1 row and not take up anymore real estate. -Why does "Email signature?" have a question mark.  It is not a question. -Filter options:  I see "X-UBC-Warning".  It is likely users won't know what this means.	Minimilist design & Gestalt theory - proximity & Match between system and real world - speak user's langauge		Low	-Shorten labels and move inputs over OR left align labels to the input fields. -Use bold very sparingly - only for those things that really need to pop. -Move "Place a copy in:" in front of the radio buttons and put both buttons on the same line. -Get rid of the "YesNo" or make it clear what it does or means. -Change "...full message headers" control to radio buttons on the same line. -Be sure to use the users terminology not system jargon
The help ? should indicate it will open in another window.  As referenced above, window management can be complex.  Users's need to know when they will be managing more than 1 window	User control & freedom - users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state quickly.		Medium		Webmail	"Open new window" notifier
While in settings, which seems to be edit mode, of the portlet, the pencil icon is still available.  Clicking on it seems to only refocus the screen so that tool bar is at top screen.	Minimalist Design - screens should not contain irrelevant information		Medium	Hide the icon when it can't be used or at minimum grey it out to show there is nothing to do with it.	Webmail
While in edit mode, the save and cancel buttons are below the fold making it hard to see them.  Not seeing the cancel button, it's natural to use the X in the top right corner to try to get out of where you are.  This is BAD since it will remove the webmail.	Error prevention & User control and freedom		Medium	Display buttons at top and bottom. Clearly have a way to  "go back" to previous view.	Webmail	"Undo or back to previous view" & Button actions on a page
Once I went to settings and then canceled out, the tab navigation disappeared along with the portlet toolbar.  I now have "settings" and "return to portal" links.  Several issues here with navigation: -The tabs should be a consistent way to move between pages. -"Return to portal" is confusing since I believe I am still in the portal.  Where the heck am I then?	Consistency & standards & Match between real system & real world - speak user's language.		Medium	Tabs should always be available. Change language to something the user will understand better -- not sure what that is right now and maybe this goes away if the tabs are persistent.	Webmail	Top level navigation (like Sakai's tabs to sites)
There is much to say about the webmail app itself but I'm not sure how much we can control.  I assume this is an outside app.  If not, it should be looked at more carefully from a heuristic perspective.
Info Finder
myWeb Bookmark's content completely changes when the icon for adding a new bookmark is clicked.  It opens the form to add a new one which is great but there is no reason to lose the high level navigation.  All the sudden it looks like something completely different and with the screen jumping that happens, it's very disorienting.	Consistency & Standards.  & Recognition versus recall		Medium
Web Search:  After typing in a search term and choosing "google search", a new window or tab opens with the search results.  There doesn't seem to be much value in having this in the page when it is as easy as opening your own new window or tab to type in search results.  Especially since this has the same window management issues referred to in several other issues...when it is unexpected.	Minimalist Design		Medium	Show search results in-line in the portlet.		Window manager?
If I am a new student, it's not clear that the 3 different "ask me" portlets would make sense to users.  Is it really necessary to have 3 different ones or could there just be one "ask me" that works like google?	Minimilist Design  & Consistency & standards (with google and search websites)		Low	Only include one "ask me" with either one search box or if 3 boxes are necessary, describe how each are different -- perhaps even, "Vancouver campus", "Okanagan campus" & "Technology questions" or something like that.
E-Learning
There are 2 webCT portlets on this page and no indication how they are different or which one I should use.  I seems like they are different because they are different colors and  have slightly different titles, "Courses" & "CoursesTESTING".  As a student, I don't know what this means.	Minimilist Design (if I don't really need both of these) & Recognition rather than recall		Medium
News
Events
Classifieds

Cognitive Walkthrough Worksheet

This section should be used by evaluators to keep track of the steps & screens they followed through the application in the cognitive walk-through. Any positive results or issues found should be included in the two sections above.

Scenario 1:  Brief description

Scenario 2:  Brief description

Scenario 3:  Brief description

Scenario 4:  Brief description