Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Sketches

Inspiring and interesting examples:

Specimen Box - an exploratory visualization & sonification tool built for Microsoft's Digital Crimes Unit by The Office for Creative Research.

https://www.youtube.com/watch?v=J0-GVIxbRPg

https://vimeo.com/49484255

http://sonification.de/handbook/media/chapter7/SHB-S7.5.mp3

http://brain.huji.ac.il/site/em.html

http://www.amedilab.com/Home/dragdrop

http://www.seeingwithsound.com/

 

Sketches-Round#1

The guessing game: The following three sketches are made based on general attributes of a pie chart, a line graph and a Venn diagram perceived by sighted users. Please listen to each sketch and try to map each sketch to one of these three diagrams. Please share your results as well as your thought process or any interesting insights:

Sketch#1.mp3

Sketch#2.mp3

Sketch#3.mp3

Answers and Received Feedback:

Sketch#1 is supposed to represent a Line Graph. It is tried to use natural sounds in this sketch to see how natural affordances help listeners identify the presented values.

  • The Drum sound is representing the X axis that is increasing e.g. 0-100.
    • Listeners assumed this was one of the values that was increasing over time.
    • Due to its rhythmic sound, the listeners were not able to clearly identify the units. 
  • The rain sound is representing the line. Volume increase and decrease indicates high and low points in the sound.
    • Most listeners identified the increases and decreases in the rain sound and associated it with a value that goes up and down.
  • The thunderstorm naturally happens when the rain is at its peak. Thus, this sound is used to represent the highest point in the line.
    • Some listeners assumed this is a separate value and not related to the rain sound.
  • Most often when the rain stops, you can hear birds singing. Thus, bird sound is used to represent the lowest points in the line.
    • Most listeners did not associate the bird sound with the rain sound and assumed that was a separate value.

Sketch#2 is supposed to represent a Pie Chart. It is tried to use sounds from the same family to represent a whole.

  • The choir pitch represents different sections of a pie chart and the length of each section represents it's relative size to the whole as well as to the other sections.
    • Most listeners were able to identify this piece as a pie chart, since there was a clear distinction between sections yet they were from the same sound family.

Sketch#3 is supposed to present a Venn diagram with three sections.

  • Each section is presented with a different type of instrument. The overlapping sounds represent the overlapping areas of the diagram.
    • This was confusing for most listeners. They were not able to identify what the overlapping sounds were indicating, specially when all three sounds were played together.
  • The metronome sound is used to show the relative size of each bubble in the Venn diagram.
    • This was very confusing for many users and assumed this piece was a line graph.

Overall feedback:

  • Adding an audio legend to the beginning of the piece
  • Trying to stay away from thematic sounds which distract the listener
  • Having time unit e.g. metronome sounds is appropriate for line graphs or bar charts not for pie charts or Venn diagrams
  • Be very careful about changing pitch and volume as visually impaired listeners could quickly pick up subtle differences

Sketches-Round#2

The following sonification sketches are based on the attached sketch to explore different ways of sonifying a Pie chart. Unlike the Round#1 Sketches, these ones are made based on a real pie chart and the timing of each value is very close to an accurate measure. A legend has been added to the piece. In the first sketch, the timing and values have 1:1 proportion, however, the timing has been cut into half for the rest of the sketches. Before listening please start with low volume and then adjust based on your preference. The goal is to spot the dominant Operating Systems market shares by listening to the following sketches.

Sonification-PieChart-Sketch#1-Legend.mp3

Sonification-PieChart-Sketch#2-Legend.mp3

Sonification-PieChart-Sketch#3-Legend.mp3

Sonification-PieChart-Sketch#4-Legend.mp3

Sonification-PieChart-Sketch#5-Legend.mp3

Received Feedback:

  • #1 and #2 -both of these make the relative proportions really clear. Prefer not to hear a pause between the segments, since you’ve got the bell indicating the start of a segment. Remove the Bell sound all together and let the segments flow to each other. It would be easier for user to listen to segments in order.
  • #3 - Difficult to differentiate between the individual sounds.
  • #4 - For the shorter/smaller segments, it may be difficult to hear it over top of all the other sounds. But it is really interesting to see how it describes the relative proportions between the segments as well as each segment relative to the whole. It would be easier for user to listen to segments in order.
  • #5 - This one really confusing - Couldn't tell why some segments overlap and others don’t, and in general couldn’t figure out what was happening.

Sketches-Round#2-Edited

Round#2 sketches have been edited based on the feedback received from the team. The following changes have been applied:

  • Sketch#1:
    • The segments are ordered from large to small to make it easier for listener to spot the more dominant segments.
    • The bell sound indicating start of a segment has been removed. The moment of silence between segments has been removed.
  • Sketch#2:
    • The segments are ordered from large to small.
    • The bell sound indicating  start of a segment has been removed.
    • The double bell sound that indicates looping is replaced by single bell sound.
  • Sketch#4:
    • The Highlighted pieces are ordered from large to small.
  • Sketch#5:
    • sounds are split into smaller pieces and are more distributed throughout the piece.

Sonification-PieChart-Sketch#1-Legend-ordered-no bell.mp3

Sonification-PieChart-Sketch#2-Legend-ordered-no bell.mp3

Sonification-PieChart-Sketch#4-Legend-ordered.mp3

Sonification-PieChart-Sketch#5-Legend-Edited.mp3

Pie Chart Tool

To start exploring how charts and graphs can be visualized, the team is going to test building a tool that can both visualize a pie chart and also create a sonic version of it. The following sketch has been proposed, and it is being reviewed by the team for further modifications.

 

Pie Chart Tool Implementation:

The following wireframes display 4 phases of the pie chart tool development. In the first phase, only the essential features are integrated. In the following phases, additional features are integrated to enable customization of the entire chart as well as its individual sections.



Sketches-Round#3

Round 3 sketches were developed based on the previous rounds’ sketches, but other features were also added. Different sounds and legend styles were used in order to reproduce the following pie chart:

Sketch #1-R3 – Audio legend is placed in the beginning; different sounds were used to represent each section, with the duration being a metaphor for size; nonstop play back.

Sonification-Sketch#1-R3.wav

Sketch #2-R3 – Audio legend is placed in the beginning of each section; different sounds were used to represent each section, with the duration being a metaphor for size; sectioned play back.

Sonification-Sketch#2-R3.wav

Sketch #3-R3 – Audio legend is placed in the beginning; only one sound is used to represent the market share, and the sections are separated by a bell sound; nonstop play back

Sonification-Sketch#3-R3.wav

Sketch #4-R3 – Audio legend is placed in the beginning; only one sound is used to represent the market share, and the sections are separated by a bell sound; sounds to indicate ten units and units were added as a metaphor for numbers; nonstop play back.

Sonification-Sketch#4-R3.wav

Sketch #5-R3 – Audio legend is placed in the beginning of each section; different sounds were used to represent each section, with the duration being a metaphor for size; sections are presented in pairs to help compare them; sectioned play back

Sonification-Sketch#5-R3.wav

Sketch #6-R3 – Audio legend is placed in the beginning of each section, but is considered part of the sonic chart; only one sound is used to represent the market share, and the sections are represented by echoed legend; a sound was added to represent ten units; the market share sound gets faster as the sections gets smaller; nonstop play back

Sonification-Sketch#6-R3.wav

Sonification Sound Tracks for Pie Chart Authoring Tool

The following sound tracks can be used to create the sonic pie chart. They are kept under 4 seconds long and made ready for looping.

2.wav

3.wav

4.wav

5.wav

6.wav

7.wav

8.wav

9.wav

10.wav

11.wav

12.wav

13.wav

Sonification Stacking Experiments

Some early and rough experiments with stacking sounds.

Strings stacked left

Strings stacked right

Strings stacked center

Experiments with Percussive Sounds and Legend Placement

Stacked percussive sounds (right justified), no legend

Stacked percussive sounds (right justified), with legend

Sequential percussive sounds with integrated legend

Sequential percussive sounds with integrated legend and "backdrop" sound

Sequential percussive sounds with legend up front

Sequential percussive sounds with single "backdrop", no legend

Some thoughts:

  • how can some of these be combined in different ways to offer the user a fuller experience of the chart/data?
  • are the pauses between percussive sounds too long? what level of accuracy do we need in terms of correlating duration with values?
  • the "backdrop" sound which indicates the full duration of the chart is quite long when it runs for the total length (sum of all parts) - in this example I have shortened it arbitrarily but made it consistent in each part such that it provides an indication of relative length of each section to total
  • if the legend is integrated, is it better to use the same sound for all parts? need to consider consistency if different presentations are combined (i.e. maintain unique sound for each section)
Section

Single sound

Single sound, integrated legend (ordered largest to smallest)

Single sound, integrated legend (duration to value).mp3

Single sound, integrated legend (frequency to value).mp3

Single sound-density to value-integrated legend.mp3

Density

Density-to-value-mapping zoomed out

Density-to-value-mapping zoom in 1X

Density-to-value-mapping zoom in 2X

Density-to-value-mapping zoom to units

  • this sample doesn't actually map density, but could provide a way of "zooming" in further on the density sample to allow the user to accurately count the units
  • maps one beep-like sound to units of 10 and same lower pitch sound to units of 1
    • each 10-unit sound is approximately 1/8sec long, each beep separated by approximately 1/4sec
    • each 1-unit sound is approximately 1/24sec long, each beep separated by approximately 1/8sec
  • unit sounds are representational, and do not relate in a durational way (i.e. sound for 1-unit is not 1/10th the length of the sound for 10-units)
  • total duration of each segment remains the sameis consistent (includes pauses at end of any section where necessary)
    • (note: because, for example, 6 1-units have a longer total duration than one 10-unit, I decided to keep the total duration of each section the same, since total duration of the sounds does not map directly to value)

Density-to-value-mapping zoom to units keychange

Stacked Right with Units - Pitch and Timbre

Stacked right, units, pitch + timbre

Stacked right, units, pitch only - strings

Stacked right, units, pitch only - clarinet

Stacked right, units, timbre only

Combinations - Stacked + Sequential

to do!