Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • FL tested with #605, 606, 607 (Early Sep)

  • FL tested with #603, 602, 608, 609 (Late Sep, early Oct)

  • These interviews are working from v1.4.0-jump.9 build of FL (currently 1.4.0-dev.54)

    • A number of changes have been made based on feedback from the Early Sep. group which has not been tested w users.

    • The fundamental issues with the experience with text alerts remain

      • These issues can be addressed with switch to explicit grab/release interaction (see “Oct (for next discussion)” below)

#603

  • PhET io sonification wrapper 1.0.0-dev.77

  • Listened to most of DOM descriptions, and self initiated movement

...

Too many alerts

...

and too long

Problem:

  1. Users not waiting to listen to whole alert - just

...

  1. listening to the first part: "magnet at center of play area"

...

  1. Lots of alerts are being fired

...

  1. (see #603 during exploration around coil

...

  1. @ ~2:00)

...

  1. . Distracting from the sonification.
  2. "Far from 4 loop coil. Very close to 2 loop coil." is too long.

Option 1:

  • Text alerts only state proximity relative to coils
  • Remove play area location from text alerts as they are redundant
  • Put closest coil first in text alert.
  • Shorten from "Far from 4 loop coil. Very close to 2 loop coil."  to "2 loop coil very close, 4 loop coil far."
  • Fire alerts only when thresholds are crossed, not on every stroke / key-up event.

Option 2:

Aside:

  • The original design was to only fire alerts when certain thresholds were passed, not with every action as it appears to be.
  • (question) We can either attempt to fix this so less alerts, or go with grab/release

  • For #603 at . In the dev version alerts are being fired regardless whether a threshold was transitioned
  • Pauses between text alert messages makes people think that's all there is.

Remove Coil to the Left / Right alert when very close to the coil

Issue:

  • The alert for "coil to left / right" can come across as jarring when already near the coil.
  • Example, some confusion for #603 after that alert "4 loop coil to the right" (2:50 - 2:54 Suddenly a different text alert appears “4 loop coil to the right”, but little context. Sudden alert makes it seem exceptional.
  • Something with this implementation doesn’t seem right, not sure why it was triggered here and not earlier.

  • ).
  • However, same user (#603) used the text cue at 36:15 and knew to move left to the coil, and again at 43:10 to find 2nd coil
    • #608 also found the prompt very helpful

Aside:

  • The original intention of this alert is to help users “set up” their magnet so they know when the coil is to the left or right.
  • (question) Perhaps this alert is not needed (user seemed to have figured it out without the alert anyway).

  • Moving magnet around - AT was reporting only play area location alerts

    • (?JH) There were supposed to be coil proximity information too. Unclear what happened to those.

When sonification and text alert are mismatched

Confusion over the coil collision sound

Problem:

  • Multiple users associated the "bong" sound with some sort of goal
    • #603 - interpreted the organ sound and the subsequent bong as a "bullseye"
    • #608 - interpreted the bong sound as "being in the coil".
  • When coil is bumped, a "bong" sound is heard, but text alert says: "magnet at center of play area". There's a disconnect with the two.
    • Example: For #603  at 2:25, magnet inside coil and hit the top of coil. "bong" sound heard, but text alert says: "magnet at center of play area".
      • Text alert creating confusion because it's not describing what the "bong" sound means
    • Example
      • 6:26 - #603 wonders why there's a different sound.
      • 6:50 - explanation was given
    • Conversely, the example of the edge bump is good because the text alert explains the sound.
  • Text alerts should prioritize coil proximity first (as mentioned by TS in Github issue comments), then magnet location.

When sonification and text alert are complimentary

Option 1:

  • Remove text alert all together and just have the "bong" sound.

Option 2:

  • When collision occurs, text alert should just say: "bumped a coil".

Aside:

  • Play area edge bump and text alert is an example where this works well: edge bump sound is accompanied by an alert saying the magnet is at the edge

Sliding Cue

Problem:

  • None of the surveyed users used the slide move (#603, #608)


Shorten the Play Area Edge Text Alert

Problem:

  • The play area edge bump text alert is a bit too long.

Option 1:

  • Change: "Magnet at middle-right edge of play area" alert seemed good. User immediately reversed directions (#603)(warning) Simplify this to just say
  • To: "At edge" or "At edge of play area", or remove all together

Option 2:

  • Remove edge bump text alert (edge bump sound may be sufficient).
  • (question) For grab/release interaction, consider adding text alerts for just the exceptional events (coil bump, edge bump).

Aside:

  • Alert seemed good. User immediately reversed directions (#603)

Tweak Circuit Mode Radio Button Text Alert

Problem:

  • Not every user listens to the full text alert when switching modes.
  • The two text alerts start off sounding the same.

Suggestion:

  • Currently:
    • "Circuit now has two different coils"
    • "Circuit now has one coil"
  • Change to:
    • "Two different coils now connected."
    • "One coil now connected".

Add coil location to scene summary and circuit description

Problem:

  • Scene summary currently lacks any information to establish the location of the coil in the play area.
  • Motivation: much of the text descriptions and alerts describe magnet and coil, and yet the coil is not mentioned at all in the scene summary.

Fix:

WhatCurrentlyProposed
Scene summary"Faraday's Law‬ is an interactive sim. It changes as you play with it.

The play area has a light bulb circuit, and a moveable bar magnet. There are controls that change what is connected to the circuit, flip the bar magnet, and reset the sim.

Move the magnet to play using arrow keys, W A S D, or 1 2 3 keys.

If needed, check out keyboard shortcuts under Sim Resources."

"Faraday's Law‬ is an interactive sim. It changes as you play with it.

At the centre of the play area is a coil* connected to a light bulb circuit. There is a also moveable bar magnet.

Move the magnet to play using arrow keys, W A S D, or 1 2 3 keys.

Additional controls that change what is connected to the circuit, flip the bar magnet, and reset the sim. If needed, check out keyboard shortcuts under Sim Resources."

*Change "is a coil" to "are two different coils" if both coils are showing.

Circuit PDOM description (1 coil)"In circuit are a light bulb, a 4 loop coil, and a voltmeter. The coil is sideways, with openings on the left and right.""Connected to the circuit are a 4 loop coil, light bulb, and a voltmeter. The coil is at the centre of the play area. The coil is sideways, with openings on the left and right."
Circuit PDOM description (2 coils)"In a circuit are a:
  • Light bulb
  • 4 loop coil
  • 2 loop coil
  • Voltmeter

The coils are sideways, with openings on the left and right."

"Connected to the circuit are a:
  • light bulb
  • 4 loop coil
  • 2 loop coil
  • volt meter

The coils are at the centre of the play area. The coils are sideways, both with openings on the left and right."

Descriptiveness

...

Concerns

  • "In 4 loop coil." ... #603 had some confusion about what that meant. (7:40)

Add "Select magnet" to the instructions for moving it

Problem:

  • Currently the scene summary and PDOM description of the magnet states they keys to move the magnet but omits the fact they need to select it first.

Suggested fix:

WhatCurrentlyProposed
Scene summary"Move the magnet to play using arrow keys, W A S D, or 1 2 3 keys.""How to play with the magnet: select magnet; move using arrow keys, W A S D, or 1 2 3 keys."
Magnet PDOM description"Use the W A S D keys to move the magnet in four directions. Use 1 2 3 keys to slide magnet left and right.""How to play with the magnet: Select the magnet; use the W A S D keys to move in four directions; use 1 2 3 keys to slide left and right.


Notes

  • #603
    • 22:55 - figures out association of sound with being in the 4 loop coil
    • 23:11 - figures out closer you get the more intense the sound
    • 25:00 - trying to figure out what the bong sound is and why it happens.
    • 29:00 - changed rate of speech, the pause throws user off. Increasing rate helps
    • Didn't feel descriptions were enough
    • Knew there was a coil and magnet, and poles.
    • 30:00 - discovers volt meter
      • unchecks it but the volt mete is still visible. something wrong?
    • 33:50 - asked what happens when magnet moved through coil at different speeds
      • doesn't know if it does anything because it just tell him that it's in the centre of the play area.
    • 35:00
      • Wasn't sure there was more than 1 way to move the coil
      • Didn't know you can press and hold
      • Looking for a change in the description as a result of moving faster, but the text alerts remain the same (issue with text alerts disconnected w sonification)
    • 36:44
      • Prompted to switch coil modes
    • 39:00-16
      • Flipping poles, but after flipping had no way of remembering the polarity. Said: "I guess that's something you just have to remember"
      • But the description is in the PDOM. Is that enough?
    • 42:00
      • Confusion over an empty Level 2 header
    • 43:00
      • Was reminded there was already a 2nd coil. User forgot and was prompted.
    • 43:30
      • Searching for 2 loop coil.
      • Text alert said "to the right" an49d used that as a cue.
    • 44:30
      • properly describes the magnet position relative to the 2 coils and knows there's a the difference between the coils because of the different bump sounds.
    • 49:00
      • Describing the sounds. Knows the more intense organ sound is important, but admits to explore it more
      • (warning) Text alerts are getting in the way.
    • 52:12
      • Asked what happens when you move magnet quickly and said they couldn't tell the difference in sound.
    • 53:00
      • Asked about the differences in coil sounds
      • Can't figure out if there's a difference. Spent so much time figuring out positioning that wasn't focusing and remembering the difference in sounds between the 2 coils.
    • 56:04
      • Identifies that the sounds in the 2 loop coil are lighter and not as deep compared to the 4 loop coil
      • Says that the sounds from the 2 loop coil "all of them" are a pitch higher.
    • 58:16
      • Asking to describe the sounds when flipping magnet, but magnet is inside the coil
      • can hear a difference between the different polarity.
    • 1:00:00
      • Asked to describe the sim as if to a friend.
      • Described it has something to do with magnet and electricity, and differences between two coils
      • Hasn't quite figured out the purpose of the N and S poles
    • 1:01:00
      • Asked whether this will help someone with impairment understand FL?
      • Answers tentatively, it's a memory thing - keep remembering what state it was previously
    • 1:02:00
      • Said it would be great if there was some way to get a quick reference "this is what it is and this is where you are"
      • You have to tab or shift-tab to find out stuff and it can be confusing.
    • 1:04:00
      • What do you like / dislike?
      • Dislikes not knowing what's going on.
      • Says they would like to be oriented "on my own". Feels like they will need to find a sighted person.
    • 1:06:00
      • Compare to other sims used previously?
      • Said having some previous experience does help. Knows to use arrow keys, knows to more through descriptions.
      • Didn't get a good sense what is happening in FL.
      • Said this one is a more complex, says he feels stupid. Still hasn't figure out what the point of the N and S poles are.
    • 1:09:00
      • (warning) Wasn't clear that the N and S poles are ON the magnet. Didn't make the association.
    • 1:10:40
      • Asked about whether the sound and descriptions worked well together
      • Says they worked well, but had to get used to the presence of the pause that was throwing them off.
      • Increasing speech rate helped.
      • Doesn't like when the tones and descriptions overlap each other - not clear
        • Doesn't think there was a problem with overlapping
        • Says can hear both description and tones comfortably - one isn't louder than the other
    • 1:12:20
      • Asked if there was a instance where the sounds and description really helped make something clear.
      • Tones and the dings with description let them know they hit something key.
      • Associates the presence of sound with getting to the "meat of things"
    • Sim's capabilities meet my requirements - Agree.
    • Using the sim is a frustrating experience - somewhat agree
    • Using the sim is easy to use - somewhat disagree
    • I have to spend too much time correcting things in the sim - strongly disagree
    • Sound were helpful - strongly agree
    • Sounds were interesting - strongly agree
    • Sounds were pleasant - strongly agree
    • Sounds were easy to understand - strongly agree
    • Sounds were relatable to their ideas - strongly agree
    • Easy to matych sounds to the meanings - strongly agree
    • It was difficult to understand how the sounds changed from one variable to the next - strongly disagree (more of a memory issue
    • Sounds were fun - somewhat agree
    • It was boring sound - disagree
    • Confusing - disagree
    • Easy to understand what the sounds mean - Strongly agree
    • The sim was complex - somewhat agree
    • The sim is easy to use - somewhat disagree
    • I need a technical support to use this - disagree
    • Functions were well integrated - somewhat agree
    • Too much inconsistency in the sim - disagree
    • I think others will learn to use this sim quickly - somewhat agree
    • I found it awkward to use - neutral
    • I felt confident using this sim - somewhat disagree

#608

  • 3:00 - appears to be having problems figuring out that focusing on the magnet is where you begin.
    • looking through the PDOM multiple times, and the Control Area, but can't find where to begin.
  • 4:10 - comments about missing something
  • 5:30
    • Asked what she was looking for, and was told to switch modes from Browse to Forms mode
    • (question) How often does this situation come up? Should instructions be put somewhere (i.e. in the Help or scene summary?)
  • 7:30
    • "I'm very confused" - can't seem to figure out how to get to magnet and move it
  •  9:40
    • Still very confused. Was manipulating the coils and not getting what she wanted. Instructed to go and find the magnet.
  • 9:48
    • Managed to find the magnet
    • (question) Is the text cue not strong enough? Just says: "Bar magnet, move in 4 directions"
  • 9:50
    • was moving magnet around, got it into the coil, bumped the coil
    • (question) Is the bump sound enough?
  • 14:20
    • Moved magnet around a lot. Bumped it along the edges a lot
    • Edge bump alert can be better.
  • 16:02
    • User slides the magnet
    • No other audio cues because far from coil
  • 18:02
    • Previously asked to turn on field lines
    • Now turns them off. Did not explore the PDOM to hear the updated description.
  • 18:40
    • Asked to describe the sim
    • Describes the sim as having a coil, a circuit, magnet, and a voltmeter (although not sure what that is)
    • Not sure what the lightbulb is for
  • 19:13
    • Knows the magnet can go in the coil because it's open on both ends
  • 19:40
    • Asked how they navigated through the sim
    • Was using the arrow keys and getting no where
      • (question) Possible the text description did not help either. The description says to use arrow keys but didn't say the context in which to use them.
    • Eventually was able to move magnet with WASD keys
  • 20:10
    • Asked if used any other keys to move the magnet
    • Said used the 1 2 3 keys and it slid the magnet, but didn't give any additional description.
  • 20:50
    • Asked about the descriptions from the screen reader
    • Said the descriptions were very specific
    • Also said didn't realize that the 1 2 3 keys wasn't sliding just inside the coil
  • 21:40
    • Asked about the amount of time spent using Browse Mode
    • Didn't know they were in Browse mode. Thought they were interacting with the sim.
  • 22:25
    • Asked what happens to the description as you move the magnet through the coil.
    • Says the descriptions tells the position relative to the coil
    • But still doesn't know what the voltmeter is for.
  • 23:00
    • Asked which coil can generate the highest voltage
    • Was confused. Didn't really recall there were two coils.
    • Thought it was very confusing
  • 24:00
    • Playing w sim again to try to better understand the two coils
    • Moving magnet around.
    • Hears the prompt for "coil to left / right" and was able to navigate to it properly.
  • 25:15
    • Hits the coil and realizes it makes a different sound.
  • 25:35
    • Asked again which coil generates more voltage
    • correctly identifies 4 loop coil generates more because it has a bigger and deeper sound.
  • 26:00
    • Asked about the polarity
    • Responded by saying the magnet goes toward the coils different
    • Says the poles repel each other.
    • User is not clear about its purpose
  • 26:30
    • Asked about the sliding
    • But said they couldn't tell the difference.
  • 28:37
    • After playing some more, still couldn't answer the question
    • Couldn't tell a difference between the movement at different speeds.
    • (question) Instructions for sliding isn't clear.
    • (question) doesn't sound like the ticking sounds for magnet movement happens during a slide.
  • 29:19
    • Asked about the sounds and what they mean.
    • Said that when the magnet is inside the coil, there's a "hitting a spring" sound.
  • 30:10
    • ASked what the clicking sound means
    • just says that the magnet is moving
  • 31:46
    • Has a strong association with the "spring hitting the table" sound.
  • 33:10
    • associates the "humming sound" with the magnet interacting with the coils.
  • 33:35
    • Asked what happens when you move the magnet through the coil
    • Says there's a "spring" sound.
  • 35:51
    • Hears the organ sound, but doesn't know what is happening
  • 36:45
    • Asked what happens when the magnet touches the coil
    • Thinks it might be the low humming sound
  • 37:40
    • After some hints, realizes that the bong sound is when the magnet touches the coil
  • 39:17
    • What happens when you flip the magnet
    • magnet makes a clicking sound and an organ sound
  • Sim's capabilities meet my requirements - Agree.
  • Using the sim is a frustrating experience - Agree
  • Using the sim is easy to use -  disagree
  • I have to spend too much time correcting things in the sim - disagree
  • Sound were helpful - strongly agree
  • Sounds were interesting - strongly agree
  • Sounds were pleasant - strongly agree
  • Sounds were easy to understand - strongly agree
  • Sounds were relatable to their ideas -  agree
  • Easy to match sounds to the meanings -  agree
  • It was difficult to understand how the sounds changed from one variable to the next -  disagree
  • Sounds were fun - strongly agree
  • It was boring sound - strongly disagree
  • Confusing - strongly disagree
  • Easy to understand what the sounds mean - agree
  • The sim was complex - somewhat disagree
  • The sim is easy to use - somewhat disagree
  • I need a technical support to use this - somewhat agree
  • Functions were well integrated - neutral
  • Too much inconsistency in the sim - strongly disagree
  • I think others will learn to use this sim quickly - somewhat agree
  • I found it awkward to use - strongly disagree
  • I felt confident using this sim - somewhat agree
  • I needed to learn new things in order to use this sim - strong disagree
  • 44:55
    • How would you describe this sim to a friend
    • "I would describe it as incredibly confusing." "There were a lot of pieces and I didn't know what to do with them until you told me."
    • Didn't like how you had to go in and out of Forms mode. "Found that incredibly frustrating."
  • 47:30
    • Would you see this sim as being helpful for others learning the concepts.
    • Said the descriptions were good and perhaps not use as much sounds.
  • 48:40
    • How would you improve the simulation
    • Label the items so they know it is interactable.
    • Thought the checkboxes were confusing
  • 49:30
    • Asked to elaborate on having more descriptions
    • Felt that the sounds and descriptions were competing with one another
    • Couldn't pay attention to both at the same time.
    • Was relying more on the description - they made more sense than the sounds did
  • 50:20
    • Compared to Friction, how does this one compare?
    • Found FL more difficult because of all the pieces and playing with each one individually.
  • 51:00
    • Asked to comment on overall experience
    • Said that once she figured out what was happening, everything became more clear
    • Found that 2 coils was more difficult than 1.
    • Liked knowing where everything was like "the coil was to the left."
  • 51:52
    • Asked about not knowing about the voltmeter was
    • Guesses that it's somehow measuring the electricity between the magnet and the coil
    • Hadn't realized that the sound was associated with the lightbulb or voltmeter

#602

  • 0:00 to 10:30
    • Exploring sim
  • 0:10:33
    • Enabled field lines and wondering about whether the field lines changed
    • Wondering what they're supposed to be showing.