Significant design issues are highlighted in green and prefixed with *. (conditions for "significant": issue has a considerable effect on experience and addressing it likely requires substantial additions/changes to the design)
Infrequently: ~ <15% of the time
Sometimes: ~ 15-35% of the time
Frequently: ~ >35% of the time
- * Frequently, there was difficulty finding objects from the virtual in the physical, and objects from the physical in the virtual.
- * Frequently, users spent more time looking at/interacting with the device than the space. This may be due to a variety of reasons, including: a) difficulty in making the physical-virtual connection (e.g., finding objects), b) slow performance of the application, and c) bias from the fact that they were user testing the mobile device. A couple of users did spend more time looking at the space than the device, however.
- Frequently, the application performance was slow. Fixed in 0.3b3.
- * Sometimes, users weren't sure what to do with the application/what the application was supposed to do. That is, some users didn't know what they should do or what the purpose of the application was. P: "Does it guide me, or do I guide it?"
- Sometimes, there was confusion about whether a particular object was in the exhibition or in storage. Users saw objects on the device but couldn't find it in the space, sometimes for catalogue, and especially artifacts under 'Related artifacts.' And if it the object is assumed to be in the exhibition, whether it's near the object currently being observed, or elsewhere in the space.
- Sometimes, there was confusion about if an object in the physical space was available in the application. Opposite issue of the above.
- Sometimes, there was confusion about whether something was a "value add" or a digital replicate. For instance, text descriptions of artifacts.
- * Sometimes, users wanted the device to guide them through the space. Some users wanted the device to tell them where to go first, and what to see next.
- * Sometimes, users felt that the device was guiding them through the space and wanted a more freeform experience. Some users felt just the opposite of the above. Specifically, they thought that the catalogue screen was meant to be experienced in order (and of those who did, one commented that it felt like a disjointed experience as the device didn't keep the same pace as the space).
- Sometimes, the keys on the onscreen keyboard were too small to type on. One user wanted a stylus.
- Frequently, there was some difficulty trying to find symbols on the onscreen keyboard. Specifically, '@' and '.'.
- Frequently, users commented that they didn't like having to scroll to the top from the bottom of the screen in order to navigate to another page. This was especially true of very long screens, such as the catalogue or artifact view with description extended, and when users wanted to go back a screen or go home.
- Frequently, SUBMIT was not the first button users tapped on to submit a text entry. Users would either tape DONE (which retracted the keyboard) then SUBMIT; or, RET, then DONE, then SUBMIT.
- Frequently, users didn't know if the application was still loading a new screen or if they hadn't tapped on an item successfully. Many users tapped several times, though some who were familiar with the iPod touch/iPhone did note the loading spinner in the status bar.
- Infrequently, users expected that tapping on 'Home' would bring the user back to the initial language selection screen.
C. Exhibitions and its subscreens
- * Frequently, exhibitions detail screen was used as the virtual 'home'. Users expected that all the mobile experience offerings for the exhibition they were in would be held in the exhibition detail screen (e.g., "Simply Montreal"). Many users went to this screen first, and stayed within this section for a large part of the reality testing.
- * Frequently, users made an unintended connection between catalogue themes and physical space sections. There was a loose link between catalogue themes and physical sections, but it was not intended that users use this as a way of navigating the space. Surprisingly, some users did find success with this.
- * Sometimes, the exhibition introduction screen (e.g., "Simply Montreal") was confusing. A few users found this screen confusing, as though they were expecting it to guide them through the space.
- Infrequently, users didn't know what exhibition they were in. A few users didn't know which exhibition to tap on in the exhibitions screen as they didn't know where they were.
- Sometimes, users didn't understand why only a few objects were shown under each theme in the catalogue.
- Sometimes, the 'View all' button in catalogue was difficult to notice. Some users found it later during the session, and noted that it wasn't easily visible (/too small).
- Infrequently, the 'Switch to grid' icon was mistaken for 'Object code entry'.
D. Object code label/entry
- * Frequently, object code entry went unnoticed/was not the obvious first choice. Corollary to point on exhibitions detail screen being using as virtual home. Many users had the instinct go to 'Exhibitions' when they first entered the space, and did not notice the 'Object code entry' option.
- Infrequently, the connection between object code label and object code entry was not made. Some users saw both the object code label and the object code entry option, but did not make a connection between the two.
- Sometimes, the object code label went unnoticed. Some users didn't even notice the object code label until it was brought to their attention by the moderator.
- Infrequently, object code label was sometimes misinterpreted. Some users thought it was a button, and tried pressing it. One expected it to read something/show a video, another expected that pressing it would add it to My Collection.
- Sometimes, users expected an 'Enter' or 'Submit' key for the object code entry. Some users were fine without it, others expected and wanted an 'Enter' button, and one or two thought that the 'Delete' button was the 'Submit' button.
- * Sometimes, it wasn't clear which object the object code label referred to. Labels were often placed near clusters of objects, and it wasn't clear which one it was referring to.
- * Sometimes, users would try entering a label number into object code entry. The label numbers were found on tombstone labels that had multiple objects on them, but they were unrelated to the object code number for the application.
- * Sometimes, having object code entry accessible from anywhere was desired. This is instead of by the current method, where one needs to go back to it, or visit it from the home screen.
- Infrequently, the expectation upon tapping on an artifact with a media badge is that it plays the media right away. Upon failure of this, the same user expected that tapping the badge itself would play the media right away.
- Infrequently, there was difficulty adjusting volume on the device. Most users did not attempt to change the volume, but some of the ones that did had difficulty knowing how to do it. Also, during the attempt, many inadvertently changed the orientation of the device.
- * Sometimes, headphones were wanted for listening to videos. Both because it was hard to hear from the device's speakers even at maximum volume, and because some users felt conscientious about disturbing other visitors/listening to something with other visitors around.
- Infrequently, users wanted to quit midway through a video and didn't know how to.
- Sometimes, users wanted to know how to get to the objects shown in the videos. Some users wanted to know if they were in the space, or added value extras.
- Infrequently, there was mild uncertainty about what 'Collect' did. One user thought it enabled purchasing of a print afterward. Almost all (including the aforementioned user) understood it was a way of storing objects for future reference.
- Infrequently, users noted that the descriptions/narratives were duplicated across platforms. Artifact descriptions were read verbatim in the videos, and extended labels had the same text as artifact descriptions.
- Sometimes, users felt that the extended artifact description was too much to read.
- Frequently, users tapped on panel expand/collapse several times because it didn't appear to do anything. The problem was that the expansion occurred below the fold and screen focus did not shift to reveal that.
- Sometimes, users didn't think to scroll down in artifact view. Some users didn't scroll down in artifact view, thus missing out on the content below the fold.
G. My Collection
- Frequently, users weren't sure if their they successfully sent their My Collection to themselves. Specifically, there was no feedback after submitting one's email for My Collection.
4. List of miscellaneous notes/observations
When a user makes a link between digital and physical, that bond corroborates that the user is on the right track. From the raw notes: Made a link between a section on the catalogue and the physical space because P saw an object in the space that she saw on the device just before / P: "Now that I know where I am, feeling a little less lost" \ [only needed a single object to feel that comfort\]
- While watching a video, users tended to spend roughly equal amount of time looking at the video and at the actual object.