Child pages
  • Uploader User Testing - Round 4 Protocol

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 5.3

...

borderfalse

...

width65%

References:

...

User Testing Protocol

...

Supporting Material:

...

Greeting script

Excerpt Include

...

User Testing Greeting Script

...

User Testing Greeting Script

User Testing Demographic Questionnaire

Scenarios

Student

Instructor

Panel

What courses are you taking that require online submission of assignments?

Wiki Markup
You are about to submit this week's assignment for \[course name\]. You have just logged into Sakai to submit this week's lab report for \[course name\].

Panel

For this task, assume you are an instructor for a Psychology 101 course site.

You just realized that you need to upload this week's lecture slides and assignment instructions for Psychology 101 to the course site.  Assume you are already in your course site for the following tasks.  You want to upload both lecture slides and assignment instructions to the file manager which is where you are.

Tasks

Panel

We're going to ask you to do a bit of role playing here.  For these tasks, we'd like you to imagine you are taking a Nutrition class this semester and your instructor has asked you to post pictures of your favorite fruit and vegetables from folders on your desktop to an online space we'll take you to in just a moment. [http://build.fluidproject.org:8080/sakai-imagegallery2-web/site/AddImages/]

Instructor

Panel

We're going to ask you to do a bit of role playing here.  For these tasks, we'd like you to imagine you are teaching a Nutrition class this semester.   It is the beginning of the semester and you'd like to show students a variety of images of foods for a discussion of their nutritional value.  You'll be putting these images from folders on your desktop onto the image gallery webpage we'll take you to in a moment. [http://build.fluidproject.org:8080/sakai-imagegallery2-web/site/AddImages/]

Tasks

Task

Description

Task 1

Please

...

choose an image of your favorite fruit from the "Fruit" folder on your desktop and put it onto the image gallery web page.

Task 2

...

Now choose images of your 5 favorite vegetables from the "Vegetable" folder on your desktop

...

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="b44bad0a-e702-4aca-a6df-ae4932e4adec"><ac:plain-text-body><![CDATA[

...

Task 3

...

Now I'd like you to upload the rest of the lecture slides, 5 - 12.  [Let them hit upload  and then...] While the files are being uploaded, you realize you didn't mean to upload lecture slides 11 & 12. What do you do?

...

]]></ac:plain-text-body></ac:structured-macro>

...

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="45c0f1bb-fe41-4036-83b6-8a8bf1a9f393"><ac:plain-text-body><![CDATA[

...

Task 4

...

[Before they hit done preferably, ask them] Now I'd like to upload 3 more of the assignments files.  Please show me how you would upload 3 more files?

...

]]></ac:plain-text-body></ac:structured-macro>

Post test questionairre

 Post test questions (after they've filled out the questionairre):
- Is there anything else you'd like to tell me about your upload experience? 
 

Notes for Test Coordinator

Offering help during the test

Try not to offer help too soon. Let the user attempt to move the image a few times.

  • If they ask for help reply with:
    • "What do you think you/that would do?"
    • Wiki Markup
      "What do you think that means \[if they ask about the icons (remove, checkmark, warning sign)\]?"

You want to observe whether users are able to:

  • recognize that they can upload files
  • select multiple files from the OS file system
  • cancel or pause upload in progress
  • remove files

Post-test questionnaire

...

width35%

...

borderColor#321137
bgColor#fff
titleBGColor#c1b7c3
titleMORE ON USER TESTING PROTOCOL
borderStylesolid

...

borderColor#CCCCCC
bgColor#F8F8F8
titleBGColor##DFDFDF
titleMORE ON USER TESTING
borderStylesolid

...

borderColor#CCCCCC
bgColor#F8F8F8
titleBGColor##DFDFDF
titleMORE ON FILE UPLOADER
borderStylesolid

...

and put them onto the image gallery web page.

Task 3

Now, please put all the images of fruit from the "Fruit" folder onto the image gallery web page.  [fluid:There should be an error message saying that some of the files have already been uploaded and asking if they should be replaced or if the action should be canceled (I'm guessing that's what happens...)]  What would you do now?

Task 4

***This is a placeholder for a task that gives the user an error message and an opportunity to cancel or retry.  We need to determine if we can complete development to make this happen.  The retry isn't implemented yet.  You can see the retry in storyboards (http://wiki.fluidproject.org/display/fluid/Uploader+Storyboard+-+Upload+with+Error)

Task 5

Now put all the images from the "Misc." folder on the desktop onto the image gallery web page.  [fluid:As soon as they start the upload, tell them...]  Actually, that was not the folder you meant to upload from/not the images you meant to upload. What would you do to stop this particular task and move onto another?

Task 6

Your last task is to put the images for three herbs and meats of your choice onto the image gallery web page. [fluid:Want to see if they user the "add more" to select from multiple folders]

fluid:Post-test Questionnaire

Have the user fill this out on their own before asking them the post-test questions below.

Post-test questions

NOTE:  Ask these after the user has filled out the post-test questionnaire.

  1. Did you notice you could upload several files at once? If so, how did you know that?
  2. Did you see any indication of the progress of your activity?
  3. What did you think of adding files from multiple folders? Was it easy to do?
  4. Could you tell how much longer the upload would take?  Was that important for you to know?
  5. Did you know when the file(s) had been uploaded successfully?  How?
  6. Did you encounter any error messages? Were they helpful in telling you what to do next?
  7. What did you like most about using this tool to upload files?
  8. What did you like least about it?
  9. Is there anything else you'd like to tell us about your experience putting images onto the image gallery page? Do you have any suggestions for how it could have worked better for you?

    Note
    titleFacilitation Tips
    • Ask the participant to think aloud.
    • Help the participant to speak freely
      • Distance yourself from the product.
      • Don't react.
      • Don't interrupt.
      • Don't draw attention to specific issues that you care about.
      • Don't ask leading questions.
      • Don't lead with your note-taking (meaning only taking notes when the participant does something interesting).
    • Don't help
    • Test the system, not the user
    • Capture themes, not every action
    • Thank your participants

    More details...