Other Modalities Yet to Explore
Haptics and Tactile Feedback
Haptics and tactile feedback could be useful for navigating, controlling, and understanding the simulation. Possible devices include: game controllers, mobile phones, and wearable vibrating motors. In the sim, tactile feedback could be used for:
- indicating when a skater has snapped to the track
- indicating proximity to the track by pulsing
Possible implementation could be done using GPII Nexus.
Using words or singing a tune, a track could be constructed using a microphone. Could incorporate nicely is there is a pre-defined "vocabulary" for different track pieces / shapes.
Other Design and Interaction Notes
- can catch the skater while it is moving, dropping, or flying.
- if skater is released right in the middle of the bowl, he/she does not move upon impact (skater used to bounce, but not anymore)
- If skater is moved close to the track, he snaps into place.
- There's a "Step forward" button, but it's unclear this is the function.
- Play pause button usable even though the skater has stopped.
- It's possible that bars in graph exceed boundaries of the chart
- Sim always starts from a "Play" mode
- Do you expect to start from paused?
Design Crit Notes
- important to capture the physics model in a non-visual form.
- friction, mass, height, speed
- sonify kinetic and potential energy
- shape of ramp sonification
- examine existing sonification of line charts (pitch is upward/downward) and control X-position to "play along"
- the skater is like a cursor - moving him around the sound changes.
- i.e. volume for proximity to ramp, notes for ramp shape
- add options for enabling / disabling audio features
Existing non-visual wayfinding:
- Most wayfinding / navigation applications for non-visual users describes location relative to the user.
- Examples: BlindSquare (Youtube video), and Sendero Seeing Eye GPS (Youtube video).
- BlindSquare has a feature which lets a user discover surroundings by panning the phone around.
- During navigation SeeingEye would use audio cues and narration to give updates and alerts (i.e. a description and a turn signal sound indicates which direction to turn).
Handisco smart white cane:
- Video on PBS about Handisco.
- Uses vibration and sound (like a Geiger counter) to convey proximity. Uses narration to give prompts and alerts.
- Smartcane video on YouTube
- Uses vibration exclusively to convey proximity to objects as to not compete with auditory information from the environment.
Model Based Sonification
- Chapter 16 in the Sonification Handbook has some interesting videos.
- Example S16.3: Tangible Data Scanning - similar to using the skater and dragging it around to scan the scene
- Example S16.4: Principal Curve Sonification of a noisy spiral data set - convey shape of a dataset (i.e. the track)
- Example S16.6: Particle Trajectory Sonification for 1 particle - conveying motion of a particle (i.e. skater). Note the difference in the quality of sound with different masses
- Chapter 17 in the Sonification Handbook
- Using k-sonar to depict shape of objects (example S17.5 to S17.11)
- Example S17.15: Smartsight square - conveying a square shape
- Chapter 20 in the Sonification Handbook
- Example S20.2: SWAN video VR demo - user navigating a physical space using sonar "pings" and earcons to confirm selections