Layout Reorderer User Testing - Round 4 Protocol
Reference: User Testing Protocol
Supporting Materials:
- Layout Reorderer demo in uPortal
- Layout Reorderer User Testing Setup for Round 4 - You'll need to reset the Welcome tab to this layout before each test!
- Consent Form
- fluid:Layout Reorderer Post Test Questionnaire
- fluid:Facilitator's test recording task sheets - use this for note taking and understanding common flows through tasks
- fluid:Task sheets to put in front of users - give these to users so they have written tasks in front of them -- give them one at time as you read it to them
- Post-test Questionnaire
Greeting script
User Testing Demographic Questionnaire
Scenarios
Student
Your university has just 'unveiled' a new portal that you'll be able to use to stay on top of campus and course information and activities. One of the features they are touting is the ability to customize your layout. In a minute, we'll show you the portal as you see it on your first visit. Since you have a few extra minutes, you decide to play around with the layout a bit.
Instructor
Your university has just 'unveiled' a new portal that you'll be able to use to stay on top of campus and course information and activities. One of the features they are touting is the ability to customize your layout. In a minute, we'll show you the portal as you see it on your first visit. Since you have a few extra minutes, you decide to play around with the layout a bit.
Tasks
Task |
Description |
---|---|
Task 1 |
Since it is the beginning of the semester and you're still getting used to your new schedule, you decide to move your course information to the top of the page. |
Task 2 |
You decide the calendar might also be pretty useful so you move it just below your course information. |
Task 3 |
Next, you decide to even-out the columns a bit and move your bookmarks to the other column. Please do so now. |
Task 4 |
You're still not exactly happy with the layout. Please adjust the layout anyway you'd like and let me know when you are done. |
Notes for Test Coordinator
You want to observe whether the user has trouble:
- recognizing they can move a portlet on the portal home page
- moving a portlet to a specific location
- differentiating between locked and unlocked portlets
fluid:Layout Reorderer Post Test Questionnaire
Note: Have users fill out this questionnaire before verbally asking them to answer the post-test questions.
Post-test questions
Note: ask these verbally after the user has filled out the questionnaire
- Did you understand how to move a box from one location to another?
- Did you notice a visual cue for picking up one of these boxes? How helpful was it to you? (looking for cursor change to hand)
- Did you notice a visual cue that helped you see where you would place a box? How helpful was it to you? (looking for green box drop target)
- Further questions about drop target, which can be skipped if answered in question #2:
- How did you know where the box you were moving would land?
- Did you notice a green outline box? What do you think it is for?
- Was there anything confusing about this box? If so, what?
- (If confusing) Are there any changes you would suggest to make it less confusing?
- Did you see a visual cue that helped you determine whether a box could be picked up or not? How helpful was it to you? (looking for lock icon to show the portlet is locked.)
- Do you have any suggestions for making any of the actions easier to do?
- Anything else you'd like to tell us about your experience changing the layout of your portal?
Blind User
- Did JAWS do a good job indicating to you the names of the boxes?
- Did you understand where to place the box?
- Did you understand if a box was locked, what that meant?
Facilitation Tips
- Ask the participant to think aloud.
- Help the participant to speak freely
- Distance yourself from the product.
- Don't react.
- Don't interrupt.
- Don't draw attention to specific issues that you care about.
- Don't ask leading questions.
- Don't lead with your note-taking (meaning only taking notes when the participant does something interesting).
- Don't help
- Test the system, not the user
- Capture themes, not every action
- Thank your participants