User Interface Options QA Test Plan - Separated Panel

User Interface Options QA Test Plan - Separated Panel

Environments

BrowserVersion
ChromeLatest Stable Release
FirefoxLatest Stable Release
MS EdgeLatest Stable Release
Safari [fluid:1]Latest Stable Release

[fluid:1] keyboard a11y can be slightly improved if you select the "all controls" option from "Keyboard Shortcuts" under the "Keyboard & Mouse" settings. May also need to use "option + tab" for tab navigation.

General QA Guidelines

General Use

  • Does the tool behave the way that you would expect
  • Are you surprised by anything
  • Does something take longer than you would expect
  • When the tool does something unexpected or takes too long to do something, does the tool provide appropriate feedback

QA Tests


ProtocolPerform the following tests using each browser/system environment

Report issues at: http://issues.fluidproject.org/secure/Dashboard.jspa

Please search for issues before reporting them, so as to limit the number of duplicate entries.

FSS Jira Filter
UI Options Jira Filter


Unit Test

Protocol

Launch the following websites to execute unit tests.

Site

State Tests

Description

Ensures that the component properly traverses through the various states. 

Protocol

Perform these tasks on the following site, perform the tests 1-13 and 14-26 in order.

Site
UI Options Demo

Mouse 

Test 1: Mouse, Open

  • Procedure
    1. Using the mouse, click to open Separated Panel UI Options
  • Expected Results
    • Separated Panel UI Options should open (slide down), with the current values selected

Test 2: Mouse, Close Immediately After Open

  • Procedure
    1. Complete Test 1
    2. Using the mouse, click the "Hide" button
  • Expected Results
    • Separated Panel UI Options should close (slide up)
    • The style of the page should not have changed

Test 3: Mouse, Change Text Style

  • Procedure
    1. Complete Test 2
    2. Using the mouse, click to open Separated Panel UI Options
    3. Using the mouse, under "text style", select a different text style
  • Expected Results
    • The text in the main window should change to the selected style

Test 4: Mouse, Change Text Size (stepper)

  • Procedure
    1. Complete Test 3
    2. Using the mouse, under "text size", click the stepper buttons to select a different text size
  • Expected Results
    • The value in the textfield should change to show the new value
    • The text in the main window should change to the selected size

Test 5: Mouse, Change Text Size (textfield)

  • Procedure
    1. Complete Test 4
    2. Using the mouse, under "text size",  click the textfield and type a new value, to select a different text size
  • Expected Results
    • The text in the main window should change to the selected size

Test 6: Mouse, Change Line Spacing (stepper)

  • Procedure
    1. Complete Test 5
    2. Using the mouse, under "line spacing", click the stepper buttons to change the line spacing
  • Expected Results
    • The value in the textfield should change to show the new value
    • The main window should change to have the selected line spacing

Test 7: Mouse, Change Line Spacing (textfield)

  • Procedure
    1. Complete Test 6
    2. Using the mouse, under "line spacing", click the textfield and type a new value to change the line spacing
  • Expected Results
    • The main window should change to have the selected line spacing

Test 8: Mouse, Change Contrast

  • Procedure
    1. Complete Test 7
    2. Using the mouse, under "colour & contrast", select a different contrast theme
  • Expected Results
    • The contrast of the content in the main window should change to the selected theme

Test 9: Mouse, Toggle Table of Contents

  • Procedure
    1. Complete Test 8
    2. Using the mouse, under "table of contents", toggle the switch
  • Expected Results
    • The display of the table of contents, in the main window, should match the selected setting

Test 10: Mouse, Enhance Inputs

  • Procedure
    1. Complete Test 9
    2. Using the mouse, under "enhance inputs", toggle the switch
  • Expected Results
    • The display of links and other inputs, in the main window, should match the selected setting

Test 11: Mouse, Close and Re-open

  • Procedure
    1. Complete Test 12
    2. Using the mouse, click the "Hide" button
    3. Using the mouse, re-open Separated Panel UI Options
  • Expected Results
    • The selected options should still be set
    • The main window should display the effects of the selected options
    • Separated Panel UI Options should display the effects of the selected options

Test 12: Mouse, Reset

  • Procedure
    1. Complete Test 11
    2. Using the mouse, click the "Reset" button
  • Expected Results
    • The options should all be set to the initial default values
    • The main window should be reset to the initial default styles
    • Separated Panel UI Options should be reset to the initial default styles
  • Stop Test
    • Refresh the browser to return the page to its initial state.

Keyboard

Test 13: Keyboard, Open

  • Procedure
    1. Using the keyboard, tap the 'tab' key until the element to open the Separated Panel UI Options button has focus
    2. Using the keyboard, tap the 'enter' key to open Separated Panel UI Options
  • Expected Results
    • Separated Panel UI Options should open (slide down), with the current values selected

Test 14: Keyboard, Close Immediately After Open

  • Procedure
    1. Complete Test 13
    2. Using the keyboard, tap the 'tab' key until the "Hide" button has focus
    3. Using the keyboard, tap the 'enter' key to close Separated Panel UI Options
  • Expected Results
    • Separated Panel UI Options should close (slide up)
    • The style of the page should not have changed

Test 15: Keyboard, Change Text Style

  • Procedure
    1. Complete Test 14
    2. Using the keyboard, tap the 'tab' key until the element to open the Separated Panel UI Options button has focus
    3. Using the keyboard, tap the 'enter' key to open Separated Panel UI Options
    4. Using the keyboard, under "text style", select a different text style
  • Expected Results
    • The text in the main window should change to the selected style

Test 16: Keyboard, Change Text Size (stepper)

  • Procedure
    1. Complete Test 15
    2. Using the keyboard, under "text size", tap the "tab" key until the textfield has focus
    3. Using the keyboard, tap the up and down arrow keys to select a different text size
  • Expected Results
    • The value in the textfield should change to show the new value
    • The text in the main window should change to the selected size

Test 17: Keyboard, Change Text Size (textfield)

  • Procedure
    1. Complete Test 16
    2. Using the keyboard, under "text size", tap the "tab" key until the textfield has focus
    3. Using the keyboard, type a new value in the textfield, to select a different text size
  • Expected Results
    • The text in the main window should change to the selected size

Test 18: Keyboard, Change Line Spacing (stepper)

  • Procedure
    1. Complete Test 17
    2. Using the keyboard, under "line spacing", tap the "tab" key until the textfield has focus
    3. Using the keyboard, tap the up and down arrow keys to select a different line spacing
  • Expected Results
    • The value in the textfield should change to show the new value
    • The main window should change to have the selected line spacing

Test 19: Keyboard, Change Line Spacing (textfield)

  • Procedure
    1. Complete Test 18
    2. Using the keyboard, under "line spacing", tap the "tab" key until the textfield has focus
    3. Using the keyboard, type a new value in the textfield, to select a different line spacing
  • Expected Results
    • The main window should change to have the selected line spacing

Test 20: Keyboard, Change Contrast

  • Procedure
    1. Complete Test 19
    2. Using the keyboard, under "colour & contrast", select a different contrast theme
  • Expected Results
    • The contrast of the content in the main window should change to the selected theme

Test 21: Keyboard, Toggle Table of Contents

  • Procedure
    1. Complete Test 20
    2. Using the keyboard, under "table of contents", toggle the switch
  • Expected Results
    • The display of the table of contents, in the main window, should match the selected setting

Test 22: Keyboard, Enhance Inputs

  • Procedure
    1. Complete Test 21
    2. Using the keyboard, under "enhance inputs", toggle the switch
  • Expected Results
    • The display of links and other inputs, in the main window, should match the selected setting

Test 23: Keyboard, Close and Re-open

  • Procedure
    1. Complete Test 22
    2. Using the keyboard, tap the 'tab' key until the "Hide" button has focus
    3. Using the keyboard, tap the 'enter' key to close Separated Panel UI Options
    4. Using the keyboard, tap the 'tab' key until the element to open the Separated Panel UI Options button has focus
    5. Using the keyboard, tap the 'enter' key to open Separated Panel UI Options
  • Expected Results
    • The selected options should still be set
    • The main window should display the effects of the selected options
    • Separated Panel UI Options should display the effects of the selected options

Test 24: Keyboard, Reset

  • Procedure
    1. Complete Test 23
    2. Using the keyboard, tap the 'tab' key until the "Reset" button has focus
    3. Using the keyboard, tap the 'enter' key to reset the options
  • Expected Results
    • The options should all be set to the initial default values
    • The main window should be reset to the initial default styles
    • Separated Panel UI Options should be reset to the initial default styles
  • Stop Test
    • Refresh the browser to return the page to its initial state.

Assistive Technology (AT)

Description
Tests to ensure compatibility with Assistive Technologies.

Protocol
Perform these tasks on the following site.

Site
UI Options Demo

Test 1: AT Tests

  • Procedure
    1. Open the browser and navigate to the specified URL
    2. Using the state tests as a guide, attempt to navigate through each state of the system
      1. Example ATs
        1. Screen Readers: JAWSNVDAVoiceOverOrca
        2. Built in AT features: WindowsMacLinuxiOSAndroid
        3. Others ATs: Speech Recognition, Screen Magnifiers, switch access, etc.
  • Expected Results
    • All states of the system should be reachable and usable while using the AT

Task Oriented Functional Tests

Description

Ensures that the component is able to handle expected input.

Protocol

Perform these tasks on the following site.

Site
UI Options Demo

Test 1: Change the Text Style

  • Procedure
    1. Open Separated Panel UI Options
    2. Under "text style", change the text style
  • Expected Results
    • The text in the main window should change to the selected style
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 2: Change the Text Size

  • Procedure
    1. Open Separated Panel UI Options
    2. Under "text size", change the text size
  • Expected Results
    • The text in the main window should change to the selected size
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 3: Change the Line Spacing

  • Procedure
    1. Open Separated Panel UI Options
    2. Under "line spacing", change the line spacing
  • Expected Results
    • The main window should change to have the selected line spacing
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 4: Change the Contrast Theme

  • Procedure
    1. Open Separated Panel UI Options
    2. Under "colour & contrast", change the contrast theme
  • Expected Results
    • The contrast of the content in the main window should change to the selected theme
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 5: Enhance Inputs

  • Procedure
    1. Open Separated Panel UI Options
    2. Under "enhance inputs", toggle the switch
  • Expected Results
    • The display of links and other inputs, in the main window, should match the selected setting
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 7: Toggle Table of Contents

  • Procedure
    1. Open Separated Panel UI Options
    2. Under "table of contents", toggle the switch
  • Expected Results
    • The display of the table of contents, in the main window, should match the selected setting
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 8: Reset Changes

  • Procedure
    1. Open Separated Panel UI Options
    2. Change some of the settings
    3. Click the "Reset" button
  • Expected Results
    • The options should all be set to the initial default values
    • The main window should be reset to the initial default styles
    • Separated Panel UI Options should be reset to the initial default styles
  • Stop Test
    • Reset the browser

Test 9: Persistence via Cookie

  • Procedure
    1. Open Separated Panel UI Options
    2. Change some of the settings
    3. Refresh the page, or change to another page and return
  • Expected Results
    • After the page is reloaded the options selected should still be in effect.
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Boundary Tests

Description

Ensures proper functionality at the input limits 

Protocol

Perform these tasks on the following site.

Site
UI Options Demo

Test 1: Minimum Text Size

  • Procedure
    1. Open the UI Options
    2. Under "text size", change the text size to the smallest value
  • Expected Results
    • The text on the main window should appear with the selected size
    • The "-" stepper should be disabled
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 2: Maximum Text Size

  • Procedure
    1. Open Separated Panel UI Options
    2. Under "text size", change the text to the largest size
  • Expected Results
    • The text on the main window should appear with the selected size
    • The "+" stepper should be disabled
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 3: Minimum Line Spacing

  • Procedure
    1. Open the UI Options
    2. Under "line spacing", change the line spacing to the smallest value
  • Expected Results
    • The line spacing on the main window should appear with the selected size
    • The "-" stepper should be disabled
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 4: Maximum Line Spacing

  • Procedure
    1. Open Separated Panel UI Options
    2. Under "line spacing", change the line spacing to the largest size
  • Expected Results
    • The line spacing on the main window should appear with the selected size
    • The "+" stepper should be disabled
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 5: Table of Contents On

  • Procedure
    1. Open the UI Options
    2. Under "table of contents", toggle the switch to "on"
  • Expected Results
    • The table of contents should be visible on the main window
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 6: Table of Contents Off

  • Procedure
    1. Open the UI Options
    2. Under "table of contents", toggle the switch to "off"
  • Expected Results
    • The table of contents should not be visible on the main window
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 7: Enhance Inputs On

  • Procedure
    1. Open Separated Panel UI Options
    2. Under "enhance inputs", toggle the switch to "on"
  • Expected Results
    • The links and other inputs should be emphasized on the main window
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 8: Enhance Inputs Off

  • Procedure
    1. Open Separated Panel UI Options
    2. Under "enhance inputs", toggle the switch to "off"
  • Expected Results
    • The links and other inputs should not be emphasized on the main window
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 9: No Preferences Changed

  • Procedure
    1. Open Separated Panel UI Options
    2. Close Separated Panel UI Options
  • Expected Results
    • After saving, the page should appear as it initially did on page load.
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 10: All Preferences Changed

  • Procedure
    1. Open Separated Panel UI Options
    2. Change all of the options
  • Expected Results
    • The main window should appear with all of the style changes
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 11: Cookies Turned Off

  • Procedure
    1. Turn off cookies in the browser
    2. Open Separated Panel UI Options
    3. Change some of the options
    4. Refresh the page, or change to another page and return
  • Expected Results
    • After the page is reloaded the options selected should no longer be in effect.
  • Stop Test
    • Refresh the page and turn cookies back on

Test-to-Fail

Description

Tests which should cause errors or not be accepted as input 

Protocol

Perform these tasks on the following site.

Site
UI Options Demo


Test 1: Enter a text size larger than the maximum

  • Procedure
    1. Open Separated Panel UI Options
    2. Under "text size", type in a text size that is larger than the maximum text size
    3. Tap the 'enter' key to set the value
  • Expected Results
    • The value should change to the maximum allowable text size
    • The "+" stepper should be disabled
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 2: Enter a text size smaller than the minimum

  • Procedure
    1. Open Separated Panel UI Options
    2. Under "text size", type in a text size that is smaller than the minimum text size
    3. Tap the 'enter' key to set the value
  • Expected Results
    • The value should change to the minimum allowable text size
    • The "-" stepper should be disabled
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 3: Enter a line spacing larger than the maximum

  • Procedure
    1. Open Separated Panel UI Options
    2. Under "text size", type in a line spacing that is larger than the maximum line spacing
    3. Tap the 'enter' key to set the value
  • Expected Results
    • The value should change to the maximum allowable line spacing
    • The "+" stepper should be disabled
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 4: Enter a line spacing smaller than the minimum

  • Procedure
    1. Open Separated Panel UI Options
    2. Under "line spacing", type in a line spacing that is smaller than the minimum line spacing
    3. Tap the 'enter' key to set the value
  • Expected Results
    • The value should change to the minimum allowable line spacing
    • The "-" stepper should be disabled
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 5: Enter non-numeric characters for the text size

  • Procedure
    1. Open the UI Options
    2. Under "text size", in the text size textfield enter non-numeric characters
    3. Tap the 'enter' key to set the value
  • Expected Results
    • The value should either not be accepted or change back to the last valid value
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Test 6: Enter non-numeric characters for the line spacing

  • Procedure
    1. Open Separated Panel UI Options
    2. Under "line spacing", in the line spacing textfield enter non-numeric characters
    3. Tap the 'enter' key to set the value
  • Expected Results
    • The value should either not be accepted or change back to the last valid value
  • Stop Test
    • Clear cookies and refresh the browser or reset the changes

Validation

Description
Tests to ensure that specifications are being met

Protocol
Perform these tasks on the following site.

Site
UI Options Demo

Test 1: HTML Validation

  • Procedure
    1. Open the browser and navigate to the specified URL
    2. For each state of the system, validate the HTML markup
      1. Tools for validation
        1. Nu HTML Checker Bookmarklet
        2. Validity Chrome Extension
        3. Lighthouse Dev Tool
  • Expected Results
    • All of the HTML markup should properly validate in all states.
  • Stop Test
    • Refresh the browser to return the page to its initial state
    • Quit the tool you are using to validate

Test 2: WCAG Validation

  • Procedure
    1. Open the browser and navigate to the specified URL
    2. For each state of the system (see: State Tests), ensure that the WCAG guidelines are being met to at least AA level
      1. Tools for validation (Note: Automated tools are not yet capable of catching all issues. It is important to go through the checklist, making use of automated tools where possible, and manual processes otherwise).
        1. WCAG AODA accessibility audit checklist
        2. tota11y Bookmarklet
        3. Lighthouse Dev Tool
  • Expected Results
    • Passes WCAG 2.0 AA requirements
  • Stop Test
    • Refresh the browser to return the page to its initial state
    • Quit the tool you are using to validate

Ad-hoc

Description
Improvised tests for quickly discovering critical issues, and uncovering ones that may be outside of formalized testing.

Protocol
Attempt to use the tool in various situations, using your imagination and freedom to explore the interface and interactions.

Site
UI Options Demo