Standardized Testing

Work in Progress...

Overview:

 

This page aims to develop design guidelines for building accessible standardized testing tools for OER (Open Education Resources) authors. 

A key feature of accessible standardized testings will be the use of laptops, desktop computers and touch screen tablets/surfaces for delivery of tests and collection of student responses. Traditionally, standardized testings have been altered extensively for students who have disabilities, however, computer-delivered accessible standardized testings aim to minimize these exceptions or special arrangements where possible and provide accommodations and supports to all students who need them without relying on teachers to provide them. *3 Computer-aided standardized testings cover a broad range of tools from automated multiple-choice type questions to the use of media, simulations and virtual laboratory spaces for testing students. Thus, where sound, vision, dexterity, the ability to spell words ‘correctly’ and speedy response times can affect the outcome of standardized testings; accessibility issues must also be considered and explored. *8

 

General Guidelines:

  • Focus on what the student is being tested for, not the mechanics of responding to a test item. You can ask the following questions to clarify the test item's intentions: *5
    • What an item is attempting to measure?
    • What elements of the item are not essential in the measurement of skill and knowledge?
    • What information can we present in alternate ways so that we can make it easier for students to access content, be engaged with it and reveal what they understand?
  • Provide text alternative (e.g. large print, braille, speech, symbols or simpler language) for any non-text content item, except in the following situation: *6
    • If non-text content is a test or exercise that would be invalid if presented in text. In this case, text alternative should provide a descriptive identification of the non-text content 
  • Make standardized testing items adaptable, so they can be presented in different ways without losing information or structure. *6
    • Make sure information, structure and relationships conveyed through presentation can be programmatically determined or available in text.
    • If the sequence in which content is presented affects its meaning, determine a correct reading sequence
    • Content should not rely solely on sensory characteristics of the component such as shape, size, visual location, orientation, or sound
  • Provide users enough time to read and use content *6
    • For items that have a time limit set by the content, enable users to turn it off, adjust it or extend it.
    • Allow students to set their own transition times between questions (but bear in mind that extra  time may make the total exam burden more onerous) *8
    • One of the exception is when the time limit is essential and extending it would invalidate the activity
  • Provide ways to help users navigate, find content, and determine where they are *6
    • Bypass blocks of text that are repeated on multiple pages
    • Include titles to describe the purpose of each page
    • Focusable components should receive focus in an order that preserves meaning and operability, if the sequence affects the meaning or operation
    • Include more than one way to locate a page within a set of pages except where the page is a result of or a step in a process
    • Enable self-selection of subtest order *7
  • If an error is automatically detected and suggestions for correction are known, then the suggestions are provided to the user, unless it would jeopardize the validity of the test *6
  • To prevent errors, for pages that submit test responses at least one of the following is true: *6
    • Submissions are reversible
    • Data entered by user is checked for input errors and the user is provided an opportunity to correct them
    • A mechanism is available for reviewing, confirming, and correcting information before finalizing the submission
     
  • Try to avoid placing great demands on certain skills such as typing, mouse navigation, and the use of key combinations which may not be accessible for many users
  •  Minimize the use of multiple screens to recall a passage since it requires greater mental efforts  
  •  Be aware that some people become more fatigued when reading text on a computer screen than on paper, specially reading long passages and text blocks 
  • The inability to see an entire problem on screen at one time is challenging for many users. 
  • The speech recognition technology may not be an accessible alternative for collecting user responses, especially for those whose native language is not English or those with speech impairments, can be frustrated by the software’s lack of ability to differentiate many of the sounds that they make. *7
  • Use of an online calculator is challenging for some students, especially if they have not had practice with this tool in their daily work. *7
  • Provide students with an option to have instructions repeated as often as they choose *7
  • Maintaining place and saving completed responses during breaks *7

  • The student must be familiar with using the technology itself. Introducing assistive technology to students with an exam deadline looming would place an added learning cost on the student *8 
  • If you use assistive technology to make the standardized testing more accessible, ask yourself if the   student is being required to learn to use new technology at short notice, and whether this constitutes fair treatment. *8
     

Functional Considerations:

  • Color should not be used as the only visual means of conveying information, indicating an action, prompting a response, or distinguishing a visual element. *6
  • Foreground and background colors can be selected by the user. *6

  • Complex backgrounds that interfere with readability of overlying text should be avoided *7

  • Except for captions and rendered text, text can be resized without assistive technology up to 200 percent without loss of content or functionality and requiring the user to scroll horizontally to read a line of text on a full screen window. *6
    • When type is enlarged on a screen, students may need to scroll back and forth, or up and down to read an entire test item. Graphics, when enlarged, may become very pixilated and difficult to view. Students who use hand held magnifiers or monocular devices when working on paper may not be able to use these devices on a screen because of the distortion of computer images. If a graphics user interface is used (versus text based), students will not have the option of altering print size on the screen. *7
  • For blocks of text, make sure the width is no more than 80 characters or glyphs (40 if CJK), text is not justified (aligned to both the left and the right margins), and line spacing (leading) is at least space-and-a-half within paragraphs, and paragraph spacing is at least 1.5 times larger than the line spacing. *6 
  • Provide alternative text or “alt tags” for images, illustrations and diagrams
  • Include accommodations such as screen magnification, text to speech audio, reverse contrast, and switch systems*3
  • Provide a script for sign language interpreters to prevent revealing some answers

  • Provide capacity to turn off monitor/ blank screen temporarily *7

  • For any moving, blinking or scrolling information that (1) starts automatically, (2) lasts more than five seconds, and (3) is presented in parallel with other content, there is a mechanism for the user to pause, stop, or hide it unless the movement, blinking, or scrolling is part of an activity where it is essential *6  
  • If any audio on a Web page plays automatically for more than 3 seconds, provide either a mechanism to pause or stop the audio, or a mechanism to control audio volume independently from the overall system volume level. *6
  • Try to avoid background audio. If you need background audio, give users control to turn it off or at least make it four times quieter than the foreground speech content (20 decibels lower) *6
  • Make all functionality available from a keyboard *6
  • When an authenticated session expires, the user can continue the activity without loss of data after re-authenticating. *6
  • Students should be able to submit their completed responses and be able to log out and back on again at another time, starting at the place where they previously left off.
    • Careful scheduling is needed for multiple test sessions to make sure that computers are available and the test security is not an issue if students who have responded to the same test items have opportunities to interact with each other between test sessions. *7
  • For any auto-updating information that (1) starts automatically and (2) is presented in parallel with other content, there should be a mechanism for the user to pause, stop, or hide it or to control the frequency of the update unless the auto-updating is part of an activity where it is essential. *6
  • When any component receives focus, it does not initiate a change of context *6 
  • Changing the setting of any user interface component  should not automatically cause a change of context  unless the user has been advised of the behavior before using the component. *6
  • Navigational mechanisms that are repeated on multiple pages within a set of pages should occur in the same relative order each time they are repeated, unless a change is initiated by the user. *6 

  • Capacity for any student to select calculator or dictionary option *7  


     

     

 

Standardized Testing Types:

TypeDefinitionContextAccessibility Issues to Consider
Low Stake Standardized TestingLow-stakes standardized testing is a form of testing encompassed by the immediate process of learning, often in a very short feedback loop, such as exercises or quizzes. Sometimes this is called "formative" standardized testing or even just "feedback". The essential characteristics are immediacy and the lack of serious consequences contingent on performance. *1
  • In classrooms
  • In groups
  • Tutoring sessions
  • Informal learning sessions
  • Providing equivalent content modalities and access mechanisms wherever possible to make the system and content flexible and adaptable for different needs and enable students to participate in these feedback loops *1
Formative Standardized Testing

The goal of formative standardized testing is to monitor student learning to provide ongoing feedback that can be used by instructors to recognize where students are struggling and address problems immediately and by students to identify their strengths and weaknesses and target areas that need work. Formative standardized testings are generally low stakes, which means that they have low or no point value. Examples of Formative standardized testings include:

  • draw a concept map in class to represent their understanding of a topic
  • submit one or two sentences identifying the main point of a lecture
  • turn in a research proposal for early feedback *2
High Stake Standardized Testing

High-stakes standardized testing has consequences that may make a serious impact on the life-course of the participant. An example might be a university entrance examination. It is important that a high-stakes standardized testing be fair to all candidates and not offer advantages to one group over another. *1

  • In classrooms
  • In groups
  • In auditoriums
  • Public venues or institutions e.g. Libraries, Examination centers, etc
  • Controlled individual kiosks/spaces
  • Group auditions
  • Individual's own setting (For projects that are submitted to the teacher online or in person)
  • Time requirements to complete the standardized testing (Addressing the automation issue)
  • Cognitive load
  • Required efforts
  • Fatigue
  • Grouping arrangements

  • Use of earphones or headphones

  • Use of individual setting if response method distracts other students

  • Availability/comparability/location of computers and peripherals

  • Glare from windows or overhead lights

  • Adaptive furniture

  • Test security
Summative Standardized Testing

The goal of summative standardized testing is to evaluate student learning at the end of an instructional unit by comparing it against some standard or benchmark. Summative standardized testings are often high stakes, which means that they have a high point value. Examples of summative standardized testings include:

  • a midterm exam
  • a final project
  • a paper

Information from summative standardized testings can be used formatively when students or faculty use it to guide their efforts and activities in subsequent courses. *2

Self Assessmentstandardized testing or evaluation of oneself or one's actions and attitudes, in particular, of one's performance at a job or learning task considered in relation to an objective standard.
  • Individual's own setting
  • At workplace
  • Accessible standardized testing tools and systems that are flexible and adapted to user's needs
  • Time requirements
  • Cognitive load

 

Standardized Testing Tools:

ToolDescriptionRequired capabilities to operate taskCurrent ExamplesAccessibility IssuesAlternative Methods
Multiple ChoiceA traditional multiple choice question (or item) is one in which a student chooses one answer from a number of choices supplied. *4
  • Cognitive
  • May not be easily accessible onscreen depending on the nature of the task (In some of the shown examples, the use of visual elements makes it inaccessible visually impaired users)
  • For visually-impaired students who rely on text-to-speech software, remembering a variety of spoken information can require considerable mental effort   *8
  • Students who rely on text-to-speech software require longer time to acquire and digest information and recognize the subtle distinctions between choices. A dyslexic student may be unable to identify such subtle distinctions at all. *8  
  • Questions that address higher-order understanding rather than surface learning are very complex and can require numerous re-readings. This has the potential to disadvantage the visually impaired student and those using text-to-speech software *8  
  • Progressing between options using the tab key (for students who cannot use a mouse) can take up a great deal of additional time. *8
  • Provide a printed version or a braille transcription (if applicable)
  • Provide text-to-speech screen readers depending on the content 
  • Provide different methods for selecting a response, such as mouse click, keyboard, touch screen, speech recognition, assistive devices to access the keyboard and select (e.g., mouth stick, head wand, one-and two-switch systems)

  • Provide an option for paper/pencil in addition to computer (e.g., scratch paper for solving problems, drafting ideas)
  • Try to avoid overly-complex answers, especially long lists with distinctions that can only be made by careful re-readings. *8
  • If using graphics, ensure that high-quality recorded descriptions are available for each graphic used or that a narrative is included with video clips to describe any action taking place. *8  
True/FalseA true-false questions is a specialized form of the multiple-choice format in which there are only two possible alternatives. These questions can be used when the test designer wishes to measure a student's ability to identify whether statements of fact are accurate or not. *4
  • Cognitive
Multiple SelectThis item allows user to choose more than one response option. *3
  • Cognitive
Labeling (Graphic)The graphic labeling task requires user to drag-and-drop labels to the correct location on an image or diagram (e.g. Venn Diagram). This type of question can be manipulated on a computer with a mouse or trackball or on a touch screen tablet. *3
  • Vision
  • Motor skills
  • Cognitive

  • Actions like dragging and dropping elements onscreen are inaccessible

  • Inaccessible to students who cannot see the screen or navigate with a mouse or touchscreen
  • Difficult to transcribe into braille or print
  • Difficult for text-to-speech screen readers
  • Inaccessible to switches
  • Replace those actions with radio buttons, matrix, or click to select interactions
  • Use of drop downs that is compatible with keyboards and switches
  • Include high-quality recorded descriptions for each graphic without disclosing the responses. 





Labeling (One to One)For this drag and drop task, the user selects a label from a list at the top of the screen and drags it to the correct location on the screen. Once the label has been used, it cannot be reused unless it is first moved back to the top of the screen. *3
  • Vision
  • Motor skills
  • Cognitive
Labeling (Unlimited)This item is similar to one-to-one labeling except that labels can be reused. The user clicks or touches the label and drags it to the correct location. *3
  • Vision
  • Motor skills
  • Cognitive
MatchingWith matching, the student matches each label with its correct response by clicking or touching one element in each column in turn. When the option in the second column is chosen, a line appears marking the connection between the two. Each option can be paired only once. *3
  • Cognitive
CategorizationThe categorization item requires the user to organize elements into categories. There are two interfaces for this item type. In the drag-and-drop interface, the user clicks or touches an element and drags it into the correct category using a mouse or touch screen. With the click-to-select interface, the user clicks or selects an element to select it. Then, the user clicks or touches the category label, and the element appears in that category. *3
  • Vision
  • Motor skills
  • Cognitive
Ordering/ RankingOrdering/Ranking/Sequencing is a drag-and-drop task in which the student must move the elements on the screen to put them in the correct order. *3
  • Vision
  • Motor skills
  • Cognitive
InteractiveIncludes videos, animations, coloring, graphing, games, puzzles or any other type of question that requires high levels of interaction between user and the interface.
  • Vision
  • Motor skills
  • Cognitive
Field SimulationsOffers simulations of real problems or exercises *4
  • Vision
  • Cognitive
MatrixThe matrix item type allows the user to assign elements to categories or groups. *3
  • Cognitive
  • Scrolling issues
  • Variations in screen size
  • Effects of magnification on graphics and table size
  • Layout issues for tables
  • High Cognitive load
  • Provide a printed version or a braille transcription (if applicable)
  • Provide text-to-speech screen readers depending on the content 
  • Provide different methods for selecting a response, such as mouse click, keyboard, touch screen, speech recognition, assistive devices to access the keyboard and select (e.g., mouth stick, head wand, one-and two-switch systems)

  • Provide an option for paper/pencil in addition to computer (e.g., scratch paper for solving problems, drafting ideas)
  • Try to avoid overly-complex Matrices, with multi columns and rows  
Assertion Reason

The assertion-reason item combines elements of multiple choice and true/false question types, and allows you to test more complicated issues and requires a higher level of learning.

The question consists of two statements, an assertion and a reason. The student must first determine whether each statement is true. If both are true, the student must next determine whether the reason correctly explains the assertion. There is one option for each possible outcome. *4

  • Cognitive

Constructed Response

(Text/ Numerical)

The text match question requires a student to supply an answer to a question or complete a blank within a brief piece of text, using words, symbols or numbers. *4
  • Motor skills
  • Cognitive
  • Implications of using the spell check when achievement in spelling is being measured as part of the writing process
 
Select Text/ Sore finger

A word/ Code/ Phrase is out of keeping with the rest of package. *4 This item requires user to select one or more words, phrases, or sentences from a passage. When user selects an option, its appearance changes to verify that it has been selected. *3

  • Cognitive
  • Implications of using a screen reader when the construct being measured is reading
  • Provide a printed version or a braille transcription (if applicable)
  • Replace those actions with radio buttons, matrix, or click to select interactions
Essays
This item requires  examination on a given topic requiring a written analysis or explanation, usually of a specified length from a sentence to multiple pages.
  • Cognitive
 
  • Difficult for students who do not use keyboard
  • Difficult for students who use speech recognition
  • Requires multiple re-readings to proofread and edit
 

 

Standardized Testing Accessible Tools:

Please test the following tools and add your comments to this page.

Tool's NameDescriptionLink
NimbletoolsNimbleTools is designed for use by general education students and students with special needs. This means that a single version of a test can be created and delivered to students across a school, district, or state in an efficient and cost-effective manner. The accessibility and accommodation tools embedded within NimbleTools also assure that students with disabilities and special needs are provided with appropriate accommodations in a standardized and controlled manner. *9http://nimbletools.com/demo/index.htm