Skip to end of metadata
Go to start of metadata
Testing protocol

Facilitator should encourage the user to think aloud throughout the testing process. The user should be given time to figure out the interactions on their own before more specific direction is provided by the facilitator.

  1. A screenshot of the previously tested, implemented PMT is used to transition to the PCP. User is instructed to close the window. 
  2. Upon closing the PMT, an instructional overlay and/or accompanying audio (spoken by the facilitator) are provided to the user. Instructions display two methods for opening a tool which provides quick access to preference adjustment (entering keyboard command or selecting the hot corner). If the instructions are not understood by the user, the facilitator explains the purpose of the PCP. Wording the user understands is made note of by the facilitator. Possible explanations:
    1. You can now open another tool which will allow you to make quick adjustments to your preferences. 
    2. Opening this tool will provide you with a way to modify your settings. 
    3. This tool will provide you with quick access to the preferences so that you can change them as you go.
  3. Depending on the method chosen by the user to open the PCP, the relevant PCP configuration is brought up:
    1. keyboard shortcut opens the key control configuration
    2. top left hot corner opens the side panel configuration 
  4. The user's initial reaction to the interface of the PCP is observed. Of particular note would be any comparisons made between the interface of the PCP and the PMT, e.g. how the preferences are presented to the user in the PCP vs. the PMT.  Any questions the user has should also be noted.
  5. At this point in the testing the user is asked what they would do in order to make an adjustment to a specific preference, or to navigate to a preference that is not immediately available. Possible facilitator prompts are listed below.  The wording understood by the user is recorded by the facilitator. Where possible, the facilitator chooses at least one option from each adjuster group below:
    1. sliders
      1. increase the size to make the text easier to read (text size - in panel and mobile configurations)
      2.  turn down the audio (volume - in panel and mobile configurations)
    2. +/- input field
      1. increase the size to make the text easier to read (text size - in key control configuration)
      2. zoom in closer (magnification level - appears only after magnifier is turned on)
    3. on/off toggle
      1. zoom in closer (magnifier)
      2. watch a video with captions (captions)
    4. radio buttons
      1. change the screen colours to white on black (contrast)
  6. The mockups reflect the live visual feedback a user would receive upon modifying a preference - if audio feedback is required, the facilitator communicates what the user would experience (e.g. audio indicates increasing volume level). 
  7. The user is now asked to close the PCP, then reopen it. User can switch between PCP configurations if they choose an alternative method of reopening the PCP. If user chooses an alternative method to open the PCP, reaction to the new PCP configuration is made note of by the facilitator. 
  8. When the user is finished adjusting preferences, they are asked "If you wish to keep these adjustments to use later on other devices, how would you do this?" - if necessary, the  question is rephrased until the user understands the purpose of the action. Wording the user understands is made note of by the facilitator. Possible alternative wording:
    1. How would you save the preferences to the cloud?
    2. How would you store all the changes you have made so you can apply them to other devices (e.g. to a public computer)?
    3. How would you keep these settings so that you can use them later on another computer?

-----

The PCP mobile configuration can either be tested after or instead of the side/key control configurations. Steps 4-6 & 8 are applicable to the mobile configuration.

 

 

Third Pilots (2015):

PCP-MM feedback wireframes

Test Protocol

Second Pilots:

> Download all mockups (ENGLISH ONLY)

PCP mockups translation - Greek

PCP mockups translation -German

PCP mockups translation - Spanish

Instructional screen 

> Download mockups

Instructional overlay appears upon closing the PMT for the first time. 

Key control configuration

> Download mockups

Opened through keyboard shortcut.

Left/right arrow keys used to navigate through preferences. Up/down arrow keys used to modify preference values.  Number values can be input for the '+/- input field' adjuster. Alternatively, user can click on the interactives (+/- buttons, arrows) to navigate/modify preferences.

Pressing tab moves focus to 'open full editor' button - selecting this button opens the PMT.  Opening the PMT, pressing the escape key, or switching focus to a different application exits the PCP. 

Side panel configuration

> Download mockups

Opened through top left hot corner. 

Mouse navigation
  • slider: handle is dragged to adjust the value - alternatively user can target and select a specific spot on the slider or select side icons to adjust the value
  • radio-buttons: clicking on a contrast theme changes the value
  • on/off toggle: clicking or dragging changes the value
  • +/- input field: clicking on +/- buttons changes the value

Clicking on another application exits the PCP. 

Keyboard navigation

User tabs through each adjuster, starting with the 'open full editor button'. 

  • slider: up or right arrow keys used to increase the value and down or left used to decrease
  • radio-buttons: up or right arrow keys used to move selection to the right, and down or left arrow keys used to move selection to the left
  • on/off toggle: hitting space or enter will toggle the value
  • +/- input field: up or right arrow keys are used to increase the value and down or left arrow keys are used to decrease the value. Number values can also be input directly.

Opening the PMT, pressing the escape key, or switching focus to a different application exits the PCP. 

Mobile configuration

> Download mockups 

Open through swiping from the bottom up. Swiping up or down loops through available preferences.
Modifying preference values
  • slider: handle is dragged to adjust the value - alternatively user can tap side icons to adjust the value.
  • radio-buttons: swiping left or right loops through contrast themes
  • on/off toggle:  tapping or dragging changes the value

Tapping or swiping outside the PCP closes the panel.

  • No labels