Skip to end of metadata
Go to start of metadata



Learning Outcomes

Primary: energy conservation

  • Bar chart is really key to showing this.
  • The pie chart is essentially showing the same information but in a different form.

Secondary: speed is related to kinetic energy

  • the only direct way to measure kinetic energy is to measure the speed of the object.
  • kinetic energy = 0.5 • mass • velocity2

Secondary: thermal energy

  • thermal energy (heat) is produced on impact and with friction, but the total energy of the system is constant.

Design Challenges

  • How do you position the skater on the simulation with a keyboard?
    • Is this possible with just native HTML controls?
    • Is this possible without having to create a custom control?
  • How would you sonify:
    • skater dropping and going down ramp (increase in kinetic energy, decrease in potential energy)
    • hitting ramp (increase in thermal energy)
    • skater going up ramp (increase potential energy, decrease kinetic)
  • If you go to the simulation with physics, how do you convey its presence and effect?
  • How do you describe the shape of the track?

The Three Energy Skate Park Scenes

There are three scenes in Energy Skate Park Basics:

  1. Introduction - demonstrates the relationship between kinetic and potential energy, as well as energy conservation.
  2. Friction - adds a friction slider to the model, otherwise plays the same as the Introduction
  3. Sandbox - free form track construction with friction and "magnetic" tracks.

Energy Charts

There are two charts, bar and pie, which show the conservation of energy, which are key to the learning outcome. The charts show:

  • Kinetic (bar + pie)
  • Potential (bar + pie)
  • Thermal (bar + pie)
  • Total (bar only)

When the simulation is running, the bars / sections of the chart animate to show the conversion of potential to kinetic energy. The pie chart follows the skater, while the bar chart stays in a static location.

Since the two charts are showing the same information but presented in different ways, it may be possible to have a single sonification of the two charts.

The thermal energy can be "thrown out" - this resets the thermal energy to zero, and decreases the total energy by the amount thrown away. After zeroing, thermal energy will continue to accumulate if there is friction. The use case for this isn't very clear but was a user feature request.


Sonification challenges:

  • How do you sonify both the bar chart and the pie chart if both are visible?
  • Since they show the same information, would you sonify only one?

Possible features:

  • Ability to choose which energy types to sonify. This would allow users to isolate just the kinetic energy and hear it rise and fall.

Bar Chart Sonification Sketch #1

Use 4 distinct sounds to the different energy types and total.

Use volume to indicate quantity. Quieter = less, louder = more.

The total energy would be a constant hum in the background that increases or decreases in volume depending on the total energy of the system.

The following sound clip is a rough rendition of a sonification of the bar graph. The funky guitar is potential energy, the deeper bass is kinetic, the bell is thermal energy, and the drum playing throughout represents the total energy.


After the skater has stopped, "return skater" buttons appear. Selecting these buttons return the skater to either the last release point, or to the bottom of the play area.

This allows the user to replay and adjust variables (i.e. friction and mass) and redo an experiment.

These buttons can overlap with elements on the page.

Using Web-Native Controls Brainstorming

Question: Is it possible to create a "drag and drop" equivalent using native web controls?

Scheme 1: Separate X and Y sliders

  • X and Y position is controlled by separate sliders.
  • For example, the X-Axis slider may be from 0 to 100. 0 = left edge, 50 = middle, 100 = right edge.
  • As the user moves across the slider, descriptions and updates can be read back so they get a sense of what is in the immediate area as well as the overall scene.

Track Shape

  • 3 basic shapes: "U", ramp, and "W" track.
  • "W" track is special because it sets up questions like "which point is potential energy the greatest / least?"
  • In free-form mode, the user can construct any shape of track by dragging sections and joining points together.
    • It's possible to create loops and other crazy tracks.


  • Describing the track shape textually
  • Using sonification to convey shape
  • How do you describe a loop shape?

Track Shape Sonification

Continuous Pitch

  • Move left and right and you can hear the shape of the track based on pitch.
  • Ascending pitch - ascent
  • Descending pitch - descent
  • Problem - This works well if the track is easily navigated left to right, but how do you convey a loop?

Sonar "Ticks"

  • The closer you get to the track, the more frequent the ticks.
  • The farther away, the less frequent the ticks.
  • You can effectively "paint" the scene by panning the mouse around the scene
  • Problem - this requires ability to use a gesture based input.
  • Problem - your perception of the track is relative to where you start painting with ticks. For example, two users with the same track may start at two different locations and get a different interpretation of the same track

Track Shape Sonification Sketch #1: Pitch, volume, and descriptions

Below: When moving the skater around to be positioned, sonification is used to indicate distance from the track below (represented by volume volume), and the elevation of the track (represented by pitch).


Below: This is a special case where the skater is being positioned within a loop.

Haptics and Tactile Feedback

Haptics and tactile feedback could be useful for navigating, controlling, and understanding the simulation. Possible devices include: game controllers, mobile phones, and wearable vibrating motors. In the sim, tactile feedback could be used for:

  • indicating when a skater has snapped to the track
  • indicating proximity to the track by pulsing

Possible implementation could be done using GPII Nexus.

Other Design and Interaction Notes

  • can catch the skater while it is moving, dropping, or flying.
  • if skater is released right in the middle of the bowl, he/she does not move upon impact (skater used to bounce, but not anymore)
  • If skater is moved close to the track, he snaps into place.
  • There's a "Step forward" button, but it's unclear this is the function.


  • Play pause button usable even though the skater has stopped.
  • It's possible that bars in graph exceed boundaries of the chart
  • Sim always starts from a "Play" mode
    • Do you expect to start from paused?

Design Crit Notes

  • important to capture the physics model in a non-visual form.
    • friction, mass, height, speed
  • sonify kinetic and potential energy
    • i.e. pitch and volume
  • shape of ramp sonification
    • examine existing sonification of line charts (pitch is upward/downward) and control X-position to "play along"
  • the skater is like a cursor - moving him around the sound changes.
    • i.e. volume for proximity to ramp, notes for ramp shape
  • add options for enabling / disabling audio features

Competitive Analysis

Existing non-visual wayfinding:

  • Most wayfinding / navigation applications for non-visual users describes location relative to the user.
  • Examples: BlindSquare (Youtube video), and Sendero Seeing Eye GPS (Youtube video).
  • BlindSquare has a feature which lets a user discover surroundings by panning the phone around.
  • During navigation SeeingEye would use audio cues and narration to give updates and alerts (i.e. a description and a turn signal sound indicates which direction to turn).

Handisco smart white cane:

  • Video on PBS about Handisco.
  • Uses vibration and sound (like a Geiger counter) to convey proximity. Uses narration to give prompts and alerts.


  • Smartcane video on YouTube
  • Uses vibration exclusively to convey proximity to objects as to not compete with auditory information from the environment.

Model Based Sonification

  • Chapter 16 in the Sonification Handbook has some interesting videos.
    • Example S16.3: Tangible Data Scanning - similar to using the skater and dragging it around to scan the scene
    • Example S16.4: Principal Curve Sonification of a noisy spiral data set - convey shape of a dataset (i.e. the track)
    • Example S16.6: Particle Trajectory Sonification for 1 particle - conveying motion of a particle (i.e. skater). Note the difference in the quality of sound with different masses
  • Chapter 17 in the Sonification Handbook
    • Using k-sonar to depict shape of objects (example S17.5 to S17.11)
    • Example S17.15: Smartsight square - conveying a square shape
  • Chapter 20 in the Sonification Handbook
    • Example S20.2: SWAN video VR demo - user navigating a physical space using sonar "pings" and earcons to confirm selections


Audio games:

  • No labels