Reference: User Testing Protocol

Greeting script

Scenarios

Student

You are a student and have just logged into Sakai to submit this week's lab report for your Biology262 course. You browse to the Biology course site.

Task

Description

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="0113cc5d-0bf3-43a2-8069-6c486ff84dbd"><ac:plain-text-body><![CDATA[

Task 1

Upload a word file from the [course name] folder on your desktop

]]></ac:plain-text-body></ac:structured-macro>

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="e5e7bee1-3e28-4477-9807-e03517014130"><ac:plain-text-body><![CDATA[

Task 2

Upload the 4 most recent files in [course name] that were saved today

]]></ac:plain-text-body></ac:structured-macro>

Task 3

While the 4 files are being uploaded, you realized two of them are the wrong files. What would you do?

Task 4

You'd like to add more files, how would you go about doing that?

Instructor

What courses are you teaching currently? Do any of them require you to upload files on the course site, such as lecture notes, instructions for assignments, pictures, etc.?

You just realized that you need to upload this week's lecture slides and assignment instructions for \[course name\] to the course site, so you have logged into Sakai. You browse to the Psychology course site and find the Add files button.

Task

Description

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="305f38da-7539-40e9-b666-b26dc7b7b96b"><ac:plain-text-body><![CDATA[

Task 1

Upload a file from the [course name] folder on your desktop

]]></ac:plain-text-body></ac:structured-macro>

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="298aefe6-c82b-400f-925a-55032268fdbd"><ac:plain-text-body><![CDATA[

Task 2

Upload the 4 most recent files in [course name] that were saved today

]]></ac:plain-text-body></ac:structured-macro>

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="3e157e82-2818-4fcd-baa9-74e018b7741c"><ac:plain-text-body><![CDATA[

Task 3

Upload all files in [course name]. <while upload progress is going> You realized that two of them are the wrong files that you didn't intend to upload. What would you do?

]]></ac:plain-text-body></ac:structured-macro>

Task 4

You remembered there are photos you wanted to add to the site from your desktop. What would you do?

Notes for Test Coordinator

Offering help during the test

Try not to offer help too soon. Let the user attempt to move the image a few times.

  • If they ask for help reply with:
    • "What do you think you/that would do?"
    • "What do you think that means \[if they ask about the red bars\]?"

You want to observe whether users are able to:

  • recognize that they can upload files
  • selecting multiple files from the OS file system
  • cancelling or pausing upload in progress
  • removing files

Post-test questionnaire

How easy or difficult was it for you to realize you could upload multiple files?

  1. Very difficult
  2. Difficult
  3. Neutral
  4. Easy
  5. Very easy

How easy or difficult was it for you to pause upload?

  1. Very difficult
  2. Difficult
  3. Neutral
  4. Easy
  5. Very easy

How easy or difficult was it for you to add more files to upload?

  1. Very difficult
  2. Difficult
  3. Neutral
  4. Easy
  5. Very easy

How easy or difficult was it for you to remove files from queue?

  1. Very difficult
  2. Difficult
  3. Neutral
  4. Easy
  5. Very easy

Post-test questions

Note: Don't read the examples in parenthesis to users; they are there to provide context for you.

1. How easy or difficult was it for you to realize you could move different sections in your portal?

  • Did you notice a visual cue for picking up one of these sections? How helpful was it to you?

2. How easy or difficult was it for you to move a section in your portal from one location to a new location?

  • Did you notice a visual cue for dropping a section? How helpful was it to you? (position, valid/invalid targets)

3. Red Bar Visual Cue:

  • How did you know where the section you were moving would land?
  • Did you notice a red bar? What do you think it is for?
  • Was there anything confusing about it? If so, what?
  • (If confusing) Are there any changes you would suggest to make it less confusing?

4. Did you have trouble moving any sections of the portal?

  • Was there a visual cue for placing a section? How helpful was it to you?

5. How easy or difficult was it for you to distinguish between sections of the portal that could be moved and those that were locked in place?

  • Did you see a visual cue that helped you determine whether a section could be picked up or not? How helpful was it to you?