Skip to end of metadata
Go to start of metadata

Two-part screen with an upper content area and a lower messaging area. More details in the text.


This is ugly looking but illustrates some design ideas I wanted to throw in to the mix before it is too late.

Description of the image:  I’m imagining this as a tablet screen.  The screen is divided into two areas.  At the top is some content, and at the bottom is a conversation area with a signing avatar and messages in the style of instant messaging.  The content at the top includes a title (“Set Up Your iPad”) in large text, a subtitle in the default size and color for the platform. The subtitle says: “This video explains some of the ways you can adjust your iPad to make it easier to use”, buttons to increase and decrease the volume, a video with a closed caption button on it, and a large ‘Continue’ button.  In the lower messaging area, the first message is a question from the avatar “Would you like me to read everything out loud?”  Yes/No buttons are attached to the right this message.  If the user does nothing, a new message from the avatar is shown and spoken: “You can answer by tapping the Yes/No buttons or by speaking”.  Next, the avatar says “Would you like to have audio description for videos?”  Again, there are Yes/No buttons, but the user could also speak.  A fourth message shown here is “You can ask a question by speaking or typing at any time”  That message has a button saying ‘Touch here for a keyboard’.  The messages would be simultaneously signed and spoken aloud as they appear on the screen.  They would appear one at a time, like an IM conversation.

Main ideas:

1.) Using an area with typical controls gives familiar-looking interaction choices to users with some experience.  If they choose to interact with these controls, we very quickly start to learn something about them and their preferences.

2.) Using a conversational interaction style would allow for flexible paths through the application that depend on previous responses.

3.) The avatar interaction would have the main focus, and is presented  in sign language and speech as well as text.  Users have several ways to respond to the avatar.  If they do not respond, or their responses are difficult to interpret, more options can be presented until something that works is found.

3.) This is still compatible with more direct manipulation selection screens, such as font size and contrast choices.  The options would appear in the top part of the screen, where users could interact with them directly, or continue in their conversation with the avatar.

4.) Just like an IM conversation, the user can take the initiative at any time, to ask a question or make a request.  For users who know what they want, this provides a shortcut.

5.) The interaction choices made by the user are also noted, and the conversation continues to use interaction methods that the user has demonstrated they can use.

6.) The conversational style provides access to prior steps in the process, and an easy way to get back to previous actions and change them.




  • No labels


  1. This is great – thanks for doing it. I have a few comments.

    1. I think it should start off with an introduction, both text and speech: "Welcome to your tablet. I'm going to ask you a few questions to help you set it up so it works best for you. You can answer by touching the screen or speaking. Let's try that. To get started, touch or say 'Continue.'" 

    2. Maybe only 1 question per screen.

    3. Maybe the first question should be "Do you want me to continue to read everything out loud?"

    4. I don't think we should have sign language as a default. It should be there as an early option, but to have it on by default may confuse some non-signing users.

  2. This 'cumulative' approach (having previous choices noted and interaction choices used as presumed preferences) is really powerful.

    And I think you're onto a great idea with the IM-like interface here. Combined with Jim's 'one q/page' comment, this could yield a very simple interface (a la Whiteney's YES/NO app screenshot) that nonetheless lets more expereinced users 'peek behind the screen' i.e. move forward through their own inquiry questions.

    Also think it makes sense to make yes/no voiceover the first question.

  3. I'm gonna comment on my comment, item #1 above. When I said "Welcome to your tablet" I was assuming too much. There are 3 other possibilities:

    • the device being used for FD is not typically used by the person doing the FD, but it's the same make/model as the one the user gets
    • not same make or model, but similar (e.g., a tablet not a desktop, so we should ask about touchscreen gestures)
    • totally different – only abstract carry-over to the user's device

    I guess there are other possibilities, such as the user having many devices, or owns no device and has to use the different ones that are available.