Simple Text Inline Edit User Testing - Round 3 Results
Summary
This round will include accessibility testing. Please note that users 1, 2, & 3 did not use assistive technologies.
Testing environment:
- User 4: MacBook running JAWS 10 on XP with VMWare
- User 5: Windows XP running JAWS 10, IE 7
- User 6: Windows XP running JAWS 7, IE 7
Demographics
User Number |
Location |
Gender |
Age |
Role |
Tech-savvy |
Do you own a personal computer? |
---|---|---|---|---|---|---|
User 1 |
University of Toronto |
Male |
25-30 |
Student |
n/a |
ibook |
User 2 |
University of Toronto |
Male |
31-35 |
Student/staff |
n/a |
ibook, ipod touch |
User 3 |
University Toronto |
Male |
31-35 |
Student |
n/a |
MacBookPro |
User 4 |
ATRC Toronto |
Male |
25-30 |
Staff |
n/a |
MacBook (Runs JAWS on VMWare XP) & EEE PC |
User 5 |
ATRC Toronto |
Male |
41-50 |
N/a |
n/a |
Pentium-4 / Windows Vista |
User 6 |
ATRC Toronto |
Female |
25-30 |
n/a |
"intermediate" |
Dell, JAWS 7, IE 7 |
User 7 |
|
|
|
|
|
|
User 8 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Do you do any of the following and if so how often?
User |
Checking email |
Instant message |
Shop online |
Online banking |
Internet research |
Take class online |
Social networking |
---|---|---|---|---|---|---|---|
User 1 |
all the time |
a few times a week |
a few times a month |
a few times a month |
all the time |
a few times a month |
all the time |
User 2 |
all the time |
a few times a week |
a few times a month |
a few times a week |
a few times a week |
never |
a few times a week |
User 3 |
all the time |
a few times a week |
a few times a month |
all the time |
all the time |
hardly ever |
a few times a week |
User 4 |
all the time |
all the time |
a few times a month |
all the time |
all the time |
hardly ever |
all the time |
User 5 |
all the time |
hardly ever |
hardly ever |
never |
a few times a week |
never |
a few times a week |
User 6 |
all the time |
never |
once a month |
few times a month |
all the time |
never |
never |
User 7 |
|
|
|
|
|
|
|
User 8 |
|
|
|
|
|
|
|
User 9 |
|
|
|
|
|
|
|
User 10 |
|
|
|
|
|
|
|
Do you upload files on the web and if so how often?
User |
Pictures |
Media |
Docs to course site |
Docs to soc. ntwkg |
File to email |
---|---|---|---|---|---|
User 1 |
a few times a month |
a few times a month |
a few times a month |
a few times a month |
a few times a week |
User 2 |
hardly ever |
hardly ever |
a few times a month |
never |
all the time |
User 3 |
a few times a month |
a few times a month |
a few times a week |
a few times a week |
a few times a month |
User 4 |
hardly ever |
a few times a month |
hardly ever |
a few times a month |
all the time |
User 5 |
a few times a month |
all the time |
a few times a week |
a few times a week |
all the time |
User 6 |
hardly ever |
hardly ever |
n/a |
n/a |
hardly ever |
User 7 |
|
|
|
|
|
User 8 |
|
|
|
|
|
User 9 |
|
|
|
|
|
User 10 |
|
|
|
|
|
What is your comfort level with technology?
User |
|
---|---|
User 1 |
not asked |
User 2 |
not asked |
User 3 |
not asked |
User 4 |
Very comfortable |
User 5 |
Very comfortable |
User 6 |
Intermediate |
User 7 |
|
User 8 |
|
User 9 |
|
User 10 |
|
If you use adaptive technologies, please also answer the following:
|
|
Browser |
How would you describe your |
When did you last upgrade your AT? |
Do you customize the default |
What level of default |
---|---|---|---|---|---|---|
User 1 |
|
|
|
|
|
|
User 2 |
|
|
|
|
|
|
User 3 |
|
|
|
|
|
|
User 4 |
JAWS 10; Voiceover 10.5.6; Orca 2.25 |
IE6 - 8; FF 3; Safari 4b2 |
Expert |
|
|
|
User 5 |
JAWS 10.0 |
IE8 |
Expert |
March 2009 |
Voice configuration, speed, key board commands, zoom |
Advanced |
User 6 |
JAWS 7. Demo versions of Speak Cue and Word Cue |
IE7 |
Intermediate |
- doesn't remember |
Voice speak. Speaks tooltips setting on. Medium verbosity. |
|
User 7 |
|
|
|
|
|
|
User 8 |
|
|
|
|
|
|
User 9 |
|
|
|
|
|
|
User 10 |
|
|
|
|
|
|
Interaction Notes
User Number |
Task 1 - Remove section |
Task 2 - Edit from LEC to DISC |
Task 3 - Edit from L to P |
Task 4 - Edit from 104 to 105 |
Task 5 - Undo |
Task 6 - Redo |
General comments |
---|---|---|---|---|---|---|---|
User 1 |
User not sure what to do at first, then clicks on remove button |
Looks for an editing tool to see if Lec is modifyable. Mouses over the section title, realizes it is editable, clicks inside, and modifies the titles. |
user changes the text quite easily |
looks for Ray Davis, and changes the section title |
User clicks inside the field and manually changes the section title again, not noticing the undo button. |
User clicks inside the field and manually changes the section title, again did not use undo/redo buttons. |
|
User 2 |
User finds and clicks the remove button successfully |
User reads the notice text at the top |
Clicks twice to the right of the P, then deletes P, then typed in L |
Users clicks three times until the cursor is found. Edits 104 to 105 easily. The clicks outside textbox. |
Notices the undo button, hovers over the undo button, and clicks undo. |
Notices the redo button and clicks redo. |
|
User 3 |
Locates the remove button and clicks it. |
|
|
|
|
|
|
User 4 |
Does a control-F to find Ray Davis. |
Ctrl-F, searched for LEC. "I have 4 sections labeled LEC, I'm going to click on the [fluid:inline edit] button even though most people wouldn't. All I did was hit Enter on the [fluid:inline edit] button, it became what appears to be an edit field. I changed the section title, and unintuitively (for a screen reader user) hit Enter . |
Backspaced the P and changed it to L. |
"One comment about the way this table is set up: There is too much information in the first column - the name of the course, assign GSIs/students. I have to hear assign GSI/Students, etc. for each row, which is more information than I need." |
Successfully made the change, but does this manually (does not notice undo). |
"I would never look for an Undo button. If I use the system all the time and know that there is an Undo button, I would use it. But there is nothing in the table that tells me that there is an Undo button. |
|
User 5 |
- Tried to select the word "Ray Davis" and pressed delete |
- has trouble discovering the field is editable (user is informed eventually that the inline edit area is editable but labelled as a button) |
Task skipped. |
Trouble getting to correct place ("P") to make change. |
- User pressed enter, going into edit mode |
- Repeats steps taken in Task 5. |
Tasks took approx. 25 minutes to complete. Some intervention by facilitator (e.g. informing user there was a 'remove' button and an inline edit area) due to the long amount of time the user was taking to find these features. |
User 6 |
Has no problem finding & deleting the row, but mentions: |
Enters "edit" mode by pressing edit; tabs to save (user has read this in instructions located above the table). |
Same as Task 2. |
Same as task 2. |
User can't find redo/undo. |
Same as task 5. |
- Table structure is difficult to navigate when there's no sense of the row/column you're in. Says this is an issue for blind users. |
User 7 |
|
|
|
|
|
|
|
User 8 |
|
|
|
|
|
|
|
User 9 |
|
|
|
|
|
|
|
User 10 |
|
|
|
|
|
|
|
Post-test Questionnaire Responses
Question |
User 1 |
User 2 |
User 3 |
User 4 |
User 5 |
User 6 |
User 7 |
User 8 |
User 9 (DS) |
User 10 (KB) |
---|---|---|---|---|---|---|---|---|---|---|
How easy or difficult was it for you to realize you could edit directly on the page (inline)? |
very easy |
very easy |
very easy |
difficult |
neutral |
[fluid:ran out of time; see User 6's "general comments" under Interaction notes.] |
|
|
|
|
How easy or difficult was it for you to edit text inline? |
very easy |
very easy |
very easy |
easy |
very easy |
|
|
|
|
|
How easy or difficult was it for you save your edits? |
very easy |
neutral |
very easy |
easy |
difficult |
|
|
|
|
|
How easy or difficult was it to know your edit was successful? |
very easy |
very easy |
very easy |
very difficult |
difficult |
|
|
|
|
|
How confident were you that your inline edit had been saved? |
very confident |
very confident |
very confident |
very unsure ? |
unsure |
|
|
|
|
|
How easy or difficult was it to undo your changes (edit)? |
very easy |
very easy |
difficult |
very difficult |
very easy |
|
|
|
|
|
How easy or difficult was it to get your changes back (edit)? |
very easy |
very easy |
difficult |
very difficult |
very easy |
|
|
|
|
|
Post-test Question Responses (verbally asked)
Question |
User 1 |
User 2 |
User 3 |
User 4 |
User 5 |
User 6 |
User 7 |
User 8 |
User 9 |
User 10 |
---|---|---|---|---|---|---|---|---|---|---|
1) Did you see anything that told you you could edit the text directly on this page? How helpful was it to you? |
I was looking for an edit link, but once I started mousing over things, I realized they could be edited. I didn't think it was intuitive, took me a little bit. |
The box above the table said the editable items would highlight. So I knew to hover around to see what can be changed. |
Not immediately. But then saw the instruction box at the top of the page. |
If I spent more time to read all the information on the page, something might have told me, but I don't want to read everything on the page. If it's a task I have to do, then I'd probably read all the information to figure out how to do it. |
When I pressed the edit button I know that I could edit. |
|
|
|
|
|
2) Did you expect that clicking the highlighted area would edit the text directly on this page? If not, what did you expect? |
|
(hovers over title) here, I know I can edit, (hovers over names) here, I assume I can't ed |
I noticed that the text box was highlighted yellow. (user didn't see the box that said "click to edit") |
|
"Yes. Because it says edit so I know I can type" |
|
|
|
|
|
3) Did you notice any indication that your edit was successful? |
I knew by the undo. When it changed, there was something that popped up and said "Edit saved". |
When I clicked outside the box, it said item saved. So I knew it was successful. |
|
|
"No." But user also mentions he hears a click sounds and that he thinks that means a save has occurred. |
|
|
|
|
|
4) Did you think there was a way to "undo" your edit? Did you think there was a way to "redo" an edit that you'd undone? |
Once I moused over the undo icon, it said Undo. Also, it wasn't here before, and it appeared once I edited |
There was an undo button. |
Yes but I did this manually. |
|
No. |
|
|
|
|
|
5) Did you notice an "undo" or "redo" link? |
Yes. The undo button is confusing. It would be better if the undo button stayed throughout the task once I've made changes. The colour of the Undo button is too strong, and I don't know if that is a good thing. |
It was kind of obvious. When you hover over them, it tells you, you can undo. |
Not immediately (not until the last task). |
No. |
No. |
|
|
|
|
|
6) Did you enjoy using inline edit? Is there anything you would improve? |
It's nice that it's easy to change. I would expect to go to an edit page. I like that it's all embedded in this page. I didn't have to go somewhere, change, save, and go back. It's also very easy to undo. |
|
I would change the red undo arrow so that it would actually say undo. |
I enjoyed using it. If there is a way to make the language more intuitive, it would work better. Users need to be taught how to use it. |
I enjoyed using it. I expected a lot of [fluid:JAWS] keyboard shortcuts to work. I had to make changes manually. |
|
|
|