Creating scenarios for user testing

The scenario describes in layman terms the task that is to be performed by the user. A good scenario will make the task explicit without leading the user to any particular path of completing the task. It may be necessary to define unique tasks for different Personas or roles.

• Try to user test the design(s) in multiple contexts (not just one context)
• make the larger context that the user is following more like a natural "story" (Note: most of the tool names below would not be given in a user testing scenario. The idea is that the user would try to figure out where to go without specifically being directed to the 'right' place. The names of the tools are there only to illustrate to user test creators where scenarios involving the pager could be found. )

For instance, if you were trying to test a 'pager' widget, you could create a scenario where the participant acts as an student who is trying to work on the latest assignment for his Biology 101 class. He finds an announcement from last week with a link to the assignment, and then finds a particular message in the forum/discussion tool to help him do the assignment. Before he submits the assignment, he reviews his grade for his past assignments in the Gradebook to try to figure out how good of a grade he needs on this assignment to be on track for an A in the course. To finish the assignment, he then needs to review some materials from another site from last semester which he's hidden, so he goes to Worksite Setup and finds the site in a somewhat but not too large list (like 30 items) of his sites.

Then the participant acts as an instructor for another scenario. First he finds the student's assignment in the drop box (haven't seen one big enough to confirm there is paging). He reads it and looks up the students' grades in the gradebook to see how he's doing overall, then enters the grade for this assignment. As the student had problem with a question that many other students had, he goes to the quiz he created in the Quiz & Survey tool (I think they page) to find out how the question looked to students. Finally, the instructor finds a site for an Accessibility group on campus he's just found out he is * interested in and wants to join in the Membership tool.

• the user works through several scenarios that all naturally flow together like they would do if using the design in the real world
• for multiple designs, have each user evaluate all of them.
  *remember to randomize the order that users work through the designs to negate any "ordering effects"

User testing scenario examples:

Announcements

1. you could create a scenario where the participant acts as an instructor and finds an announcement from last week with a link to the assignment,
2. (Student) There was an announcement last week about a logistics change to the midterm.  You want to check out the announcement again to make sure you're clear on what to do tomorrow for the midterm.

Forums

1. finds a particular message in the forum/discussion tool,
2. (Student) You are required to respond or begin 2 discussion threads each week.  Your instructor starts a couple discussions each week and fellow students have also created new discussions. Choose 2 threads of interest and respond to them. (As far as set-up, I think you'd want to assume we're several weeks into the semester so there are lots of posts (with dates appropriate)).
3. (Instructor) You require you're students to respond to or create 2 discussion threads each week. You start the week off with a couple of questions on this weeks topic. It's now the end of the week and you need to check on students participation (this work counts toward their participation grade).

Membership Tool

1. finds a particular site they want to join in the Membership tool
2. An instructor (or student) recently found out there are joinable sites in Sakai. They heard someone talking about this site with great campus resources but they don't know the name.  Ask them to go to the Membership tool and browse (browse is probably too leading so maybe another term) join any sites that are of interest.

Worksite Setup

1. finds one of their sites in a somewhat but not too large list (like 30 items) in Worksite Setup,

Gradebook

1. finds a particular student's grades in the gradebook,
2. You've just finished grading student's essays from last week. Enter their grades in your gradebook. (I'd give them a stack of essays with handwritten grades on them -- this is how most faculty use it).

Drop box

1. finds a particular student's assignment in the drop box (haven't seen one big enough to confirm there is paging).

Email Archive

1. (Instructor) Find an email [student's name] sent last week so you can respond.

Quiz & Surveys

1. find a particular question in a quiz (I think they page) they created in the Quiz & Survey tool.

One thing that is important to understand are the variations (different contexts, attributes of the list, etc.) that require different pagers. Since we know we need to deliver multiple pagers, the design pattern will be critical in helping people understand when and shy various pagers apply (and of course why use paging at all). Hopefully some of this will come out in testing along with the research and competitive analysis you've done.

Creating taks for user testing