Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 5.3

User Testing Goals

Section
borderfalse
For File Uploader user test we want to consider ease of use for uploading files to Sakai \[to start with\].  Research questions/goals Our access to participants is limited primarily to students over faculty.    We will try and include a few faculty.    We want to test a range of non-technical and more tech savvy students as well as faculty members of the University of Toronto and UC Berkeley.    For our user test with working prototype we will have 5-7 users \ [fluid:combination of students and faculty, but primarily students\].

** If doing: For earlier paper-prototyping sessions we will work with a smaller number of participants.  

Designs 

File Upload Designs

Protocol

 Method and test coordinator script for the user test.

Greeting the User

Column
width65%
Wiki Markup

Status

Completed April 2008 by Erin Yu and Daphne Ogle

Summary

High-level summary to be added after testing has been completed.

Notes

Goals

For File Uploader user test we want to consider ease of use for uploading files to Sakai [fluid:to start with].

Excerpt

Research questions/goals include:

  • Do users realize they can upload files?  Do they realize they can upload multiple files?
  • Are users successful at uploading a file in general?  A single file from their computer?  Multiple files from their computer?
  • Can a user find a particular file easily?
  • Do users recognize and understand intuitively how to upload files?
  • Is pause/resume upload understandable or does this confuse them
  • Does the user understand that you cannot "cancel" the upload until you first "pause" the upload? Is differentiating between paused and not paused states understandable to users?
  • Are visual affordances meaningful and helpful to users?

User Test

Success Criteria

A successful design has been achieved when:

  • 80-90% of users can successfully upload a file from their desktop.

Protocol

Method and test coordinator script.

Users

The File Uploader will be used by a wide cross-section of students and faculty in higher educational institutions.  We'd like to include keyboard-only users in the user test if possible (can prototype handle keyboard nav?).

Wiki Markup
Wiki Markup
Hi \[user's name\].  I'm \[your name\] with the Fluid team.  Today we are looking for ways to improve the usability of a Fluid component.  Remember we are not testing you.  If you find something difficult to use, chances are that others will as well.  If you feel uncomfortable you can stop at any time during the study.

Please speak all your thoughts aloud as you go through the tasks.  This helps us better understand why you are making certain choices.  The study will take about 45 minutes.  We will answer any questions you have at the end of the study.

Do you have any questions? If not then, let's get started!  <sound enthusiastic>

First we'll need you to sign this Consent Form .

Wiki Markup
Next we'll need you to complete this brief demographic questionnaire [Demographic Questionnaire|Demographic Questionnaire] \[questionnaire linked here is only for students\].&nbsp; All your information will be kept private.&nbsp; Your name will not be associated with the results at all.&nbsp;&nbsp;

Scenarios

Note: Trying to create scenarios where participants will use the File Upload component without us telling them specifically "to upload a file".

Scenario 0

You are taking an Introduction to Photography class. Your current assignment is to take three pictures demonstrating different types of lighting and share them with the rest of the class for discussion and critique. You found and took several pictures with your digital camera and transfered them to your computer several days ago. You now need to locate those pictures and submit them for the assignment to the class Image Gallery. If you remember correctly, you transferred the pictures to your computer into a folder named "photography 101".

Scenario 1

It's the middle of the semester and your group project for Psychology 101 is due in two days.  You are collaborating with several other students.  Each of you has taken a section of the project to work on.  As each team member finishes their section they IM the group to let everyone know their progress. 

Your job on the project team is to take all the sections and combine them into one document and submit to the instructor through your class Sakai site (folder: "final project"). 

The files you need to combine are:
- introduction.doc
- method.doc
- results.doc

The files may be located in any of the following places:
- folder "class project" on the computer's hard drive
- Psychology 101 class work space on Sakai
- Sociology 100 class work space on Sakai

Scenario 2

You are a TA for undergraduate class "Psychology 101" and teach a tutorial/discussion section of 30 students.  You are responsible for lecturing in the tutorial as well as grading the assignments of your 30 students. 

Last week was the course mid term and you have just finished grading all 30 online assignments.  Your feedback remarks for each student are written in a Word document.  You need to get each student's feedback to them on their space on the course Sakai site.  You are hoping to do this for the following three students' feedback.

Student Files:
- Allison_feedback.doc
- Colin_feedback.doc
- Daphne_feedback.doc

Scenario 3

You are an instructor for undergraduate class "Psychology 101" and teach a large lecture of 300 undergraduate students in the Fall term.  There are typically 10 tutorial/discussion sections of 30 students each taught by your TAs.  You have your TAs do the grading of exams and assignments.  It is the Winter term/Fall semester and recently you have been asked by your department chair to offer this course in the summer term/session as an online course.  In preparation for the new online course you have created a separate Sakai site for it and started organizing your lectures and notes by week.

Today you are working on the materials for the new online course for week 3.  You have a lecture in Powerpoint and 2 short assignment descriptions that you have been finalizing.  You have just recruited a few new TAs and want them to familiarize themselves with these materials and so are making the materials available on the new course Sakai site.

Files for TAs to review are:
- lecture_3.ppt
- Assignment_3.doc
- Group_assignment_description.doc

Notes for Test Coordinator

Offering Help during the test

Try not to offer help too soon.  Let the user attempt to upload a file a few times.
* If they ask for help reply with:
** "What do you think you/that would do?"
** "What do you think that means?"

You want to observe whether the user has trouble:
* uploading a file from the computer, another site, or current Sakai site
* is pause/resume upload understandable or does this confuse them
* does the user understand that you cannot "cancel" the upload until you first "pause" the upload, is this confusing
* differentiating between paused and not paused states
* is the wizard layout for the Continue and Cancel buttons confusing? should these two buttons be moved adjacent to each other instead of separated on either side of the panel?

After the Test

* Questions for the Test Coordinator to ask the user after the tasks are completed.  (Note: Don't read the examples in parenthesis to users; they are there to provide context for you.)
* If you are using Likert scales, have the user complete those before asking them questions.  

1. How easy or difficult was it for you to add the files?

2. How easy or difficult was it for you to select files from another Sakai site?

3. How easy or difficult was it for you to Pause during file upload and then Resume the upload?
•    Can you explain more about your thoughts during this operation?

4. How easy or difficult was it for you to Cancel adding a file?

Likert Scales

How easy or difficult was it for you to add the files?

|----------------------|---------------------|----------------------|-------------------|
very difficult              difficult              neutral                   easy                 very easy

How easy or difficult was it for you to select files from another Sakai site?

-----------------------|---------------------|----------------------|------------------|
very difficult              difficult            neutral                easy                 very easy

How easy or difficult was it for you to Pause during file upload and then Resume the upload?

-----------------------|---------------------|----------------------|------------------|
very difficult              difficult              neutral               easy                 very easy

How easy or difficult was it for you to Cancel adding a file?

-----------------------|---------------------|----------------------|------------------|
very difficult              difficult            neutral                easy                 very easy

 Results

Wiki Markup
Here are the results of our user testing on the File Uploader \[link to results\]

Interaction Design

Underlying design patterns and description of component behavior.

Test Environment

Location and version of the environment that was used. Attach a screenshot of the environment at the time of testing if the environment will change over time.

Results

Full notes and analysis of the user tests.

Uploader User Testing - Round 1 Results

Excerpt Include
Uploader User Testing - Round 1 Results
Uploader User Testing - Round 1 Results
nopaneltrue

More...

Column
width35%
toc
Panel
borderColor#321137
bgColor#fff
titleBGColor#c1b7c3
titleTable of Contents
borderStylesolid
borderStylesolid
titleMORE ON USER TESTING
Recently Updated
fluid
fluid
labelsuser_testing
label = "file_upload"
Panel
borderColor#566b30#93BB6F
bgColor#fff#F8F8F8
titleBGColor#D3E3C4titleRelated Links
borderStylesolid
Content by Label
file_uploadfile_upload
showLabelsfalse
maxResults20
showSpacefalse
cql
titleMORE ON FILE UPLOAD
Recently Updated
fluid
fluid
labelsfile_upload