Introduction and Setup
At OpenEd 2015 in Vancouver, there was an opportunity to conduct two informal, impromptu user tests on the Forces & Motion Basics simulation.
User 1 is an executive of an accessibility technology organization and has no vision, and User 2 is a university has low vision and can perceive general shapes by wearing an aid. Evaluations were done in Firefox on Windows with JAWS on their personal laptops.
The following simulation was used: http://www.colorado.edu/physics/phet/dev/html/forces-and-motion-basics/1.1.5-accessible-instance.3/forces-and-motion-basics_en.html?accessibility&screens=1
Testing was limited to navigating the interface without actually playing the simulation (i.e. no pullers were placed on the ropes). Playing of the simulation was not done because of time restrictions. Evaluations took about 30-45 minutes including discussions.
- None of the users used "Application Mode" for JAWS. Each user approached the simulation as they would a website, and employed strategies they were accustomed to.
|The simulation lacks standard landmarks used for screen reader navigation||One user approached the simulation by asking JAWS to list any available headers as a way to give an outline of the content. When that came back empty, they then attempted to list any tables, forms, and lists. All of those came back empty. The user then started to go through the content line by line by "arrow-ing" down (Ctrl + arrow key).||Add a H1 header for the simulation title. Use a <summary>, <details>, or simply a <p> to describe the scene.|
|Checkboxes for Sum of Forces and Values missing labels.|
- Currently it just reads back "checkbox unchecked".
- One user misinterpreted the checkboxes as being related to the Go and Return buttons. The Go and Return buttons were read back as disabled, and the user thought that since the checkboxes came immediately afterwards, checking the boxes it would enable them. Labels would help clarify this.
|The simulation lacks a description||One user was hoping for some guidance in terms of what to anticipate "on the page".||Use a <summary>, <details>, or simply a <p> to describe the scene.|
|Pullers have roles as graphics. It was not clear that they were interactive.||A few times the pullers were focused, but the user did not know they can activate it. Both users deduced they were cosmetic and ignored them.|
- Add an interactive role to them. Perhaps role="button"?
- Add a short instruction to the description of the puller. "Large puller. Select to put on rope."
|Two small pullers have the same name. This creates confusion.||Both users were not sure why there are two "small pullers". Wasn't sure if it was an error or if there really was two.|
|Give the two small pullers different names. Or call them "Small Puller #1" and "Small Puller #2".|
|Puller labels are read back twice. The second time reads back very quickly - like it's lacking punctuation.||User encountered the repeating labels - found it more curious than confusing.|| |
|JAWS reveals elements that should be hidden to the screen reader||Toward the bottom of the document, one user was having JAWS read out a long floating point number, even though such a number was not visible on the screen.||Seems to be a div at the end of the document with a floating point number. This should have aria-hidden="true" to hide it from ATs.|
|"Return" button needs a better description||One user remarked that "Return" button was confusing. Wasn't sure what it was returning to.||Use an "aria-labelledby" to give a better label for ATs.|
|Disabled "Go" and "Return" buttons may require better descriptions when disabled||One user became determined to get the Go and Return buttons working.|| |
|Sound button lacks feedback||When enabling disabling the sound, there's little feedback that indicates it did anything.|| |
|By arrowing down through the content, one user was able to have the screen reader read back a name starting with "Sean".|| || |
|Unless user knows who "PhET" is, they may get confused.||Both users did not know what the term "PhET" referred to. After hearing the word read, one user went character by character spelling out the word - which did not help (was unfamiliar with the term).|
Provide two "phet" logos for sighted and non-sighted users. For the "sighted" logo, put
aria-hidden="true" to hide it from ATs. For the "non-sighted" logo use this technique outlined on WebAim to make invisible content for screen readers.
Example implementation: http://handbook.floeproject.org/. See the logo inside the
|It is not clear what "puller" means||Neither user had any idea what a "puller" is. One user thought it was a machine, or maybe something related to a pulley.|| |
|It is not clear what "group" means||Both users had no context for what the group is about, or why it's there. Not sure of its importance.|| |
|JAWS reads the pullers on separate lines giving a false sense they are arranged in columns.|
|Pullers are presented on separate rows, giving a spatial impression of information organized in a column. But this contradicts the descriptions of the pullers who are arranged horizontally "left" and "right"||User suggested to put the pullers in a table. See "Pullers in a Table" for further discussion.|
|Unclear what is the importance of left and right || ||More context setting / descriptions will help with understanding the groupings.|
|Document <title> should be more concise.||Right now the title is "Forces and Motion: Basics 1.1.5-accessible-instance.3". One user remarked: "I have no idea what that is."|| |
Pullers in a Table
One user suggested to arrange the pullers in a table thus giving them a left-to-right orientation which matches the descriptions and expectations. They demonstrated this navigation by creating a 1 row by 4 column table in Microsoft Word and put the words "left large", "left medium", "right medium", and "right large" into each cell.
I added second row at the start and labeled each cell "Empty slot" without telling the user what I had done. The user then navigated through and immediately understood that something needed to go into those spots and perhaps that's what the pullers are for.
Point of discussion:
- How can we re-arrange the pullers so an AT reads them back in a way that implies a left-right orientation? Currently a list does not provide this.
- Is a table the ideal structure or is there a better alternative?
- Both users had different ways of navigating a page. User one preferred to have JAWS summarize the content using headers, tables, forms etc. The second user preferred to tab through first.
- Neither user used Application Mode in JAWS
- One user tried to OCR the simulation screen using JAWS' built-in OCR feature.