Child pages
  • Layout Reorderer User Testing - Round 3 Results

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 5.3

Summary

Excerpt
  • 4 of 4 users were able to drag and drop portlets. The one user who took a longer time to realize she could do this (and she did eventually do it) was confused by the task.
  • 1 user suggested that when mousing over a movable portlet it might be helpful for it to say "click & drag to move."
  • 4 of 4 users understood what a locked portlet was
  • Performance was markedly improved

Demographics

User Number

Location

Gender

Age

Role

Tech-savvy

Do you own a personal computer?

User 1 (BF)

Berkeley

Female

41-50

Staff

Medium

Yes, Macbook

User 2 (AL)

Berkeley

Male

19-24

Student

Medium-high

Yes, Toshiba laptop (PC)

User 3 (JV)

Berkeley

Female

19-24

Student

Medium

laptop, windows XP

User 4 (DB)

Berkeley

Female

51-60

Staff

Medium-low

Yes, a PC

User 5







User 6







User 7







Do you do any of the following and if so how often?

User

Checking email

Instant message

Shop online

Online banking

Internet research

Take class online

Social networking

User 1 

All the time

All the time

All the time

All the time

N/A

A few times a month (webinar)

No answer

User 2 

All the time

A few times a month

A few times a month

A few times a month

A few times a week

Hardly ever

A few times a week

User 3

All the time

A few times a month

Hardly ever

A few times a month

A few times a week

Never

A few times a week

User 4

All the time

Hardly ever

A few times a month

All the time

A few times a month

A few times a month

A few times a week

User 4








User 5








User 6








User 7 








Do you upload files on the web and if so how often?

User

Pictures

Media

Docs to course site

Docs to soc. ntwkg

File to email

User 1 

Hardly Ever

Never

Never

Hardly Ever

All the time

User 2 

A few times a month

Hardly Ever

Never

Hardly Ever

A few times a week

User 3

A few times a week

A few times a month

Hardly ever

Never

All the time

User 4

A few times a week

Hardly ever

Never

Hardly Ever

All the time

User 5






User 6






User 7






Interaction Notes

User Number

Previous LMS Experience

Task 1 - Move a particular box/portlet

Task 2 - Change the placement of other boxes/portlets

Task 3 -  Deal with a Locked box

Task 4 (if not done already) - drag and drop a portlet

General comments

User 1


The Announcements portlet wasn't there as it should have been, so we had the user move the Weather portlet. User seems to have thought we wanted her to move it to another tab, as she said "I never customize the tabs." Daphne had to explain that we wanted her to "move that box" to make it clear what we wanted. She had no problem dragging and dropping the box, but still didn't think that's what we wanted. She didn't understand at first that when you moved a box to a new spot, the box underneath it moved down--she was looking for this box for a bit and couldn't find it.

She had no trouble dragging and dropping portlets, but she still wanted to interact with the tabs. She tried "Add content" and "Add tab." She said "I would put my bookmarks under a tab; I wouldn't put them up front."

She dragged the portlet way up to the upper left and read the pop-up message. Then she said, "What's considered a locked box?" She realizes "Welcome" is locked when she sees the lock icon and quickly realizes she can't move the Bookmarks portlet up any higher because of it.

She had no problem moving the image lightbox down.


User 2


Scrolls down to Weather and drags and drops it with no problem. Says it's similar to the Facebook interface.

Tries to drag Weather up. Moves Bookmarks to the left. Moves Image Gallery to the right and there's a little weirdness when he tries to move it to a space where there is no portlet - he can't drop it.

Tries to drag there two times, then quickly sees that it's locked. Tries to click on the word "Welcome" in the portlet header to unlock it.

Easily performs this task.


User 3


Easily drags and drops the portlet right away. Tries to put it on a locked portlet and fails so moves it to upper right. Says right away, "I didn't see this lock so I tried to put this on top of it. Now the lock makes sense."

Calendar drags a little jerkily as she moves it up, but she still has no problem moving it.

She'd already done this in task #1 so we didn't ask again.

She moves Weather down across/below a somewhat bit portlet with no problem.


User 4


Initially asks, "is that considered a folder?" Then says, "no, it's an object on the page. Shouldn't there be something that tells me I can move things?" She sees "Page Layout." Then she says, "Can I move it (the portlet)? I *can* move it maybe?" Sees the red box and drops it and says, "I can!" Realizes she can and moves the portlet pretty quickly from start to finish.

Easily drags Calendar between Weather and Bookmarks portlets.

User tries to drag the portlet to the upper-left. Says, "it won't change with the Welcome so that's the top-most left."

User easily moves the Weather portlet to the left.

Thought it might be helpful when mousing over something for it to say "click & drag to move."

User 5







User 6







User 7







Post-test Questionnaire Responses

Question

User 1

User 2

User 3

User 4 

User 5

User 6

User  7

How easy or difficult was it for you to realize you could move boxes in the portal by dragging a box from one location and dropping it in a new location?

Neutral

Very easy

Very easy

Difficult




How easy or difficult was it for you to actually move a box in the portal (by dragging and dropping) to a specific location?

Very easy

Easy

Very easy

Very easy




How easy or difficult was it for you to distinguish between boxes in the portal that could be moved and those that were locked in place?

Neutral

Easy

Very easy

Easy




Post-test Question Responses (verbally asked)

Question

User 1

User 2

User 3

User 4

User 5

User 6

User 7

User 8

User 9 (DS)

User 10 (KB)

1) Did you notice a visual cue for picking up one of these boxes? How helpful was it to you? (looking for cursor change to hand)

Didn't notice a visual cue. Did see borders on a box come up when she started to drag and drop.

He says he didn't notice a visual cue because "you said pick up the box" which told him he could do that. Says the bars on the top of the portlets are modeled "kind of like Windows." Also, the bar at the bottom of the portlet gives it sort of a 3-D feel, which makes him think he can move it.

"I just assumed you could (drag and drop) because you can do that on Google things." She didn't notice a visual indicator.

First things she saw were "Customize" or "Page Layout," like in MyYahoo! and a couple other places and thought that's what we wanted. (Though she says maybe MyYahoo! will allow you to drag and drop, too -- she doesn't know.)
When I clicked I saw the four-cornered arrow and when I see that I feel that I can move something. It would have been easier if it had said "you can click & drag." The box also changed to an outline, which I think means it's active and I can do something with it.







2) Did you notice a visual cue for dropping (and placing) a box? How helpful was it to you? (looking for green box drop target)

She noticed the shadow of the box when the image is moving with you. Didn't realize it would move things down, however.

The translucent view was a clue. I've seen that at deviantart.com. After thinking about it, he says, "I guess what helps with placement is the pink box." "It's sort of like solitaire--you pick up a card and move it."

"Yeah, I like the red boxes. It makes it very clear."

I should have read that Welcome is locked, I see it's locked now. I just moved it elsewhere and everything shifted around it. I felt comfortable about placement...after I tried to move it where it wouldn't go, I understood where things would go.







3) Further questions about drop target, which can be skipped if answered in question #2:

  • How did you know where the box you were moving would land?
  • Did you notice a green outline box? What do you think it is for?
  • Was there anything confusing about this box? If so, what?
  • (If confusing) Are there any changes you would suggest to make it less confusing?

It was confusing because I wasn't sure what the task was. It would have been helpful to tell her she was "customizing a web page for my own information... "I didn't realize that's what we were doing." I'm not sure how you'd add tabs.










4) Did you see a visual cue that helped you determine whether a box could be picked up or not? How helpful was it to you? (looking for lock icon to show the portlet is locked.)


The "combination lock thing."

"It made sense; the box is locked. I can't move it." Said she saw nothing else on the screen (the message) that told her she couldn't move it.

When I let it go and it went down below Welcome  I was nervous (at the beginning of the test) so I didn't notice the lock.






 

Potential Design Improvements (based on testing)

  • There still seems to be a slight issue if you try to drag a portlet into whitespace way below other portlets--it can't be dropped until you drag it close to the bottom portlet on a column. You should be able to drop anywhere in that whitespace.