Documentation for a historical release of Infusion: 1.3
Please view the Infusion Documentation site for the latest documentation.
If you're looking for Fluid Project coordination, design, communication, etc, try the Fluid Project Wiki.

Simple Text Inline Edit User Testing - Round 3 Results

Summary


This round will include accessibility testing. Please note that users 1, 2, & 3 did not use assistive technologies.
Testing environment:

  • User 4: MacBook running JAWS 10 on XP with VMWare
  • User 5: Windows XP running JAWS 10, IE 7
  • User 6: Windows XP running JAWS 7, IE 7

Demographics

User Number

Location

Gender

Age

Role

Tech-savvy

Do you own a personal computer?

User 1

University of Toronto

Male

25-30

Student

n/a

ibook

User 2

University of Toronto

Male

31-35

Student/staff

n/a

ibook, ipod touch

User 3

University Toronto

Male

31-35

Student

n/a

MacBookPro

User 4

ATRC Toronto

Male

25-30

Staff

n/a

MacBook (Runs JAWS on VMWare XP) & EEE PC

User 5

ATRC Toronto

Male

41-50

N/a

n/a

Pentium-4 / Windows Vista

User 6

ATRC Toronto

Female

25-30

n/a

"intermediate"

Dell, JAWS 7, IE 7

User 7







User 8





 







 







 


Do you do any of the following and if so how often?

User

Checking email

Instant message

Shop online

Online banking

Internet research

Take class online

Social networking

User 1 

all the time

a few times a week

a few times a month

a few times a month

all the time

a few times a month

all the time

User 2 

all the time

a few times a week

a few times a month

a few times a week

a few times a week

never

a few times a week

User 3

all the time

a few times a week

a few times a month

all the time

all the time

hardly ever

a few times a week

User 4

all the time

all the time

a few times a month

all the time

all the time

hardly ever

all the time

User 5

all the time

hardly ever

hardly ever

never

a few times a week

never

a few times a week

User 6

all the time

never

once a month

few times a month

all the time

never

never

User 7








User 8








User 9








User 10








Do you upload files on the web and if so how often?

User

Pictures

Media

Docs to course site

Docs to soc. ntwkg

File to email

User 1 

a few times a month

a few times a month

a few times a month

a few times a month

a few times a week

User 2 

hardly ever

hardly ever

a few times a month

never

all the time

User 3

a few times a month

a few times a month

a few times a week

a few times a week

a few times a month

User 4

hardly ever

a few times a month

hardly ever

a few times a month

all the time

User 5

a few times a month

all the time

a few times a week

a few times a week

all the time

User 6

hardly ever

hardly ever

n/a

n/a

hardly ever

User 7






User 8






User 9 






User 10






What is your comfort level with technology?

User


User 1 

not asked

User 2 

not asked

User 3

not asked

User 4

Very comfortable

User 5

Very comfortable

User 6

Intermediate

User 7


User 8


User 9 

User 10


If you use adaptive technologies, please also answer the following:


User


What type of AT do you use?
(List technology & version)

Browser
(Name & version)

How would you describe your
AT skills?

When did you last upgrade your AT?

Do you customize the default
settings of your AT?

What level of default
setting do you use?
|

User 1 







User 2 







User 3







User 4

JAWS 10; Voiceover 10.5.6; Orca 2.25

IE6 - 8; FF 3; Safari 4b2

Expert




User 5

JAWS 10.0

IE8

Expert

March 2009

Voice configuration, speed, key board commands, zoom

Advanced

User 6

JAWS 7. Demo versions of Speak Cue and Word Cue

IE7

Intermediate

- doesn't remember
- upgrade every 5 years because that's when she gets an AT subsidy from the government

Voice speak. Speaks tooltips setting on. Medium verbosity.


User 7







User 8







User 9 






User 10







Interaction Notes

User Number

Task 1 - Remove section

Task 2 - Edit from LEC to DISC

Task 3 - Edit from L to P

Task 4 - Edit from 104 to 105

Task 5 - Undo

Task 6 -  Redo

General comments

User 1

User not sure what to do at first, then clicks on remove button

Looks for an editing tool to see if Lec is modifyable. Mouses over the section title, realizes it is editable, clicks inside, and modifies the titles.

user changes the text quite easily

looks for Ray Davis, and changes the section title

User clicks inside the field and manually changes the section title again, not noticing the undo button.

User clicks inside the field and manually changes the section title, again did not use undo/redo buttons.




User 2 

User finds and clicks the remove button successfully

User reads the notice text at the top
"I mouse over, and it looks to me like I can highlight it."
User clicks inside the field, deletes the existing text (LEC), and puts in DISC.
"That's cool I can edit. I hover over it, and you see it's an editable thing." 
Users highlights from left to right with a mouse (click and drag) then edited the text.

Clicks twice to the right of the P, then deletes P, then typed in L

Users clicks three times until the cursor is found. Edits 104 to 105 easily. The clicks outside textbox.  

Notices the undo button, hovers over the undo button, and clicks undo.


Notices the redo button and clicks redo.


User 3

Locates the remove button and clicks it.







User 4

Does a control-F to find Ray Davis.

"I anticipate tabular data, and the headings are being read. There is a column heading called Remove. Underneath the Remove, I'm getting 'True' buttons. To me, 'True' doesn't communicate what the button will do. Should say 'Remove this course, etc.'
I'm going to click on the True button. It removed that row. It didn't tell me it got removed. I just stayed in the same row in the table, the next row came up so I just assumed I did. Nothing validated that I got that course row removed.
There was no confirmation message such as 'Do you want to remove This Course at this time at this place' so I know what is being removed and that I'm focused on the right item..."

Ctrl-F, searched for LEC. "I have 4 sections labeled LEC, I'm going to click on the [fluid:inline edit] button even though most people wouldn't. All I did was hit Enter on the [fluid:inline edit] button, it became what appears to be an edit field. I changed the section title, and unintuitively (for a screen reader user) hit Enter .
I've never had a web site that worked properly like a desktop app...

after the edit has been made, whatever is written there, I'm not getting it. It's a blank session. It's not even an inline edit button. I had to refresh the JAWS virtual buffer to get the information in the row back.

I'm going to edit the second last item now. I had to hit enter twice to get the edit.

Once again, I need to refresh the JAWS buffer to get the changes I made to show up.

Backspaced the P and changed it to L.

"One comment about the way this table is set up:  There is too much information in the first column - the name of the course, assign GSIs/students. I have to hear assign GSI/Students, etc. for each row, which is more information than I need."

Successfully made the change, but does this manually (does not notice undo).

"I would never look for an Undo button. If I use the system all the time and know that there is an Undo button, I would use it. But there is nothing in the table that tells me that there is an Undo button.
I would do a Find on the page "Undo""
Does a JAWS "find" but it does not discover the Undo.


User 5

- Tried to select the word "Ray Davis" and pressed delete
- Used shift-space to select the whole row and pressed delete
- When unable to delete, coordinator informed user that there was a remove button
- User notes that there is a button, but it is saying "true"; user eventually deletes row
- User notes that in Yahoo! and other websites, it says "delete" or "remove"
- Didn't know that the row was successfully deleted though; there was no response or reaction from the interface

- has trouble discovering the field is editable (user is informed eventually that the inline edit area is editable but labelled as a button)
-after being informed of this user can edit area
- Not sure how to save changes:  Initially tried to use ctrl-s to try to save the edit area
- then went  into the browser file menu, noted that the save option was disabled
 - User was not aware that it was saved
- User asked if the option to save was displayed at all on the monitor (e.g., like a button), or whether it was under the menu bar (tried to find it there, couldn't)
-    Would like an option to save it
-    Would like some indication on how to save the edit

Task skipped.

Trouble getting to correct place ("P") to make change.
-Not sure how to save changes
-User found the editable area
-pressed F2 (likely in an attempt to try and edit), which didn't work
-User somehow lost his way after that and had difficulty finding the editable area again
- Once he found the editable area and semi-successfully edited (edited the wrong words), he pressed tab, which saved the field. This was done inadvertently, not intentionally.
- User asked coordinator if there was a save button
- Didn't know he needed to press enter to save (no indication to do so)
- Enter edits the field; thought it just jumps back and forth between modes, not necessarily save?

- User pressed enter, going into edit mode
- Erased 104, typed 107 back manually
- Pressed enter to save, now knowing that enter saves the field
- User notes that no indication was given on whether it was saved or not
- When asked to revert from 107 back to 104, wasn't sure what mode he was in, tried using arrows to move
-undo button not perceivable

- Repeats steps taken in Task 5.
- redo button not perceivable

Tasks took approx. 25 minutes to complete. Some intervention by facilitator (e.g. informing user there was a 'remove' button and an inline edit area) due to the long amount of time the user was taking to find these features.

When it was described how the inline edit should interact ("This is labelled as a button, but you can edit the area"), the user grasped the functionality immediately. This suggests that the user needs an orientation to how to interact with the component.

Despite encountering several barriers, the user's overall impression of the component was positive.

Confusion over how to save. E.g. assumes saving means going to IE File > Save.

User 6

Has no problem finding & deleting the row, but mentions:
"The delete button should say 'delete row'. Also after a delete the user needs to know what was deleted."

Enters "edit" mode by pressing edit; tabs to save (user has read this in instructions located above the table).  

Same as Task 2.

Same as task 2.

User can't find redo/undo.

Same as task 5.

- Table structure is difficult to navigate when there's no sense of the row/column you're in. Says this is an issue for blind users.
-"I know I press tab to save but how do I know if it's saved?"
- Mentions that both the "remove row" and inline edits are buttons. Stresses - "how do users know which buttons they can edit with vs. the buttons they can delete with? What if someone tries to edit a "delete row" button, so they press enter, but this inadvertantly deletes a row?

User 7








User 8







 

User 9








 

User 10










Post-test Questionnaire Responses

Question

User 1

User 2

User 3

User 4 

User 5

User 6

User  7

User 8

User 9 (DS)

User 10 (KB)

How easy or difficult was it for you to realize you could edit directly on the page (inline)?

very easy

very easy

very easy

difficult

neutral

[fluid:ran out of time; see User 6's "general comments" under Interaction notes.]






How easy or difficult was it for you to edit text inline?

very easy

very easy

very easy

easy

very easy






How easy or difficult was it for you save your edits?

very easy

neutral

very easy

easy

difficult






How easy or difficult was it to know your edit was successful?

very easy

very easy

very easy

very difficult

difficult






How confident were you that your inline edit had been saved?

very confident

very confident

very confident

very unsure ?

unsure






How easy or difficult was it to undo your changes (edit)?

very easy

very easy

difficult

very difficult

very easy






How easy or difficult was it to get your changes back (edit)? 

very easy

very easy

difficult

very difficult

very easy






Post-test Question Responses (verbally asked)

Question

User 1

User 2

User 3

User 4

User 5

User 6

User 7

User 8

User 9 

User 10 

1) Did you see anything that told you you could edit the text directly on this page? How helpful was it to you?

I was looking for an edit link, but once I started mousing over things, I realized they could be edited. I didn't think it was intuitive, took me a little bit.
When I highlighted them, they turned yellow.

The box above the table said the editable items would highlight. So I knew to hover around to see what can be changed.

Not immediately. But then saw the instruction box at the top of the page.

If I spent more time to read all the information on the page, something might have told me, but I don't want to read everything on the page. If it's a task I have to do, then I'd probably read all the information to figure out how to do it.

Since it does say 'button', I'd be adventurous .


When I pressed the edit button I know that I could edit.
[note: after many unsuccessful attempts to edit the facilitator informed the user that inline edit was labeled as a button]


[fluid:ran out of time; see User 6's "general comments" under Interaction notes.]





2) Did you expect that clicking the highlighted area would edit the text directly on this page? If not, what did you expect?


(hovers over title) here, I know I can edit, (hovers over names) here, I assume I can't ed

I noticed that the text box was highlighted yellow. (user didn't see the box that said "click to edit")




"Yes. Because it says edit so I know I can type"






3) Did you notice any indication that your edit was successful?

I knew by the undo. When it changed, there was something that popped up and said "Edit saved".

When I clicked outside the box, it said item saved. So I knew it was successful.


It said item saved.


No.

"No." But user also mentions he hears a click sounds and that he thinks that means a save has occurred.






4) Did you think there was a way to "undo" your edit? Did you think there was a way to "redo" an edit that you'd undone?

Once I moused over the undo icon, it said Undo. Also, it wasn't here before, and it appeared once I edited

There was an undo button.

Yes but I did this manually.


No. Even after searching the page for the term Undo.

No.





 

5) Did you notice an "undo" or "redo" link?

Yes. The undo button is confusing. It would be better if the undo button stayed throughout the task once I've made changes. The colour of the Undo button is too strong, and I don't know if that is a good thing.

It was kind of obvious. When you hover over them, it tells you, you can undo.

Not immediately (not until the last task).

No.

No.





 

6) Did you enjoy using inline edit? Is there anything you would improve?

It's nice that it's easy to change. I would expect to go to an edit page. I like that it's all embedded in this page. I didn't have to go somewhere, change, save, and go back. It's also very easy to undo.


It's a nice way to edit.

I would change the red undo arrow so that it would actually say undo.
The remove (error)
symbol made more sense to me.

I enjoyed using it. If there is a way to make the language more intuitive, it would work better. Users need to be taught how to use it.
And  the inline edit field does not behave like a button - but it is labeled as a button.

I enjoyed using it. I expected a lot of [fluid:JAWS] keyboard shortcuts to work. I had to make changes manually.

JAWS should read every button. E.g.: Save, remove. It should indicate what the button does (not true). If I press enter, it should say ("save"). there should be a save button that's indicated.

No sound/click to know you can edit.  If you press E it should jump to where it can edit.

CTRL Z as undo - I wish that would let you undo a change.

When I go to MS Word and I save, JAWS says, "save". Why not in inline edit? I couldn't find a save button here.






Potential Design Improvements (based on testing)