User testing tests the usability of a component by putting it in front of several users individually to see if they can use the component successfully. A user testing protocol describes the objective(s), methodology, and organization of a user test - essentially how to administer a user test.

These guidelines provide a template for a user testing protocol, which may then be tailored to specific protocol for user testing of components. Scenarios, tasks, post-questionnaires, and post-test questions must be defined on a per-test-set basis.

Greet user and explain user testing procedure

The over-arching goal of the user greeting is to make the person feel comfortable. Making the person comfortable produces more accurate test results. In order to make the person comfortable:

• Be at ease yourself (practice beforehand).
• Introduce yourself and any other observers or note takers.
• Establish an understanding of the test purpose and goal.
• Explain user testing procedure and that you are testing the system and not them.
• Set expectations for the test length and scope.
• Provide an escape route (that they can stop at any time).
• Ask them if they have any questions.
• Convey a positive and enthusiastic attitude.

Greeting script

Sample greeting script for use in user testing.

Have user sign consent form

A consent form is usually necessary to ensure that the user affirms (in writing) willingness to participate in the study. Have the user sign the consent form, and give address if an honorarium has to be mailed.

Ask if user has any questions

Before starting the study, pause and let the user ask questions. Respond as needed to ensure that the user is at ease and ready to participate.

Have user complete the demographic questionnaire

Have the user fill out the demographic questionnaire. If you would like to build rapport with the user, you can also ask the questions verbally, perhaps beginning with an ice breaker question about some of the demographic information you want to collect anyway, to help put the user at ease.

Conduct the test

• Have the user sit down in front of the computer (or paper prototype).
• Remind the user that you would like them to 'think aloud' so that you can understand what they are thinking about their experience.
• Read the user the scenario for context.
• Read the user the task to complete. If the tasks are complex, you may want to put a sheet of paper in front of the user with each task written out (as your goal is not to test the user's ability to remember the task).
• Record the user's actions, comments, questions, and body language.

Offering help during the test

Try not to offer help too soon. Let the user make multiple attempts at completing the task.

If they ask for help reply with:

• "What do you think you should do?"
• "What do you think that would do?"
• "What do you think that means?"

Record responses, and note if you needed to provide help in order for the user to complete the task.

Have user complete a Post-test Questionnaire (optional)

In order to gather more information after the test is complete, have the user complete a short questionnaire. This is an optional step. Post-test questionnaires often use questions with Likert scale responses.

Post-test Questionnaire example question

How easy or difficult was it for you to...?

1. Very difficult
2. Difficult
3. Neutral
4. Easy
5. Very easy

Post-test questions (optional)

In order to gather more information after the test is complete, have the user answer reflective questions. This is an optional step. Be sure this step is after they've completed the post-test questionnaires to not bias their answers.

Post-test question examples

1. How easy or difficult was it for you to...?
2. Did you notice a visual cue for...?
   • What do you think it is for?
   • How helpful was it to you?
3. How did you know to...?
4. Was there anything confusing about...?
   • If so, what?
   • (If confusing) Are there any changes you would suggest to make it less confusing?

Conclude the test

1. Let the user ask any questions and debrief.
2. Let the user know how they can find out about changes made to the component as a result of the user testing data.
3. Thank the user for participating.
4. Give the user an honorarium if you have one.
   • If the honorarium will be mailed let the user know when to expect to receive it.
   • Give the user a contact number in the case the honorarium doesn't arrive.