Skip to end of metadata
Go to start of metadata

User research needs to cover a broad and diverse range of users.  We want to make sure we are hearing from a comprehensive set of users in Higher Education Campuses.  In order to limit scope and allow us to gain understanding at various levels -- from user's goals (what drives them) to the details of a particular workflow we will complete the contextual inquiries in iterations.  Our initial focus will be on schools close to home for UC Berkeley and U Toronto where the research teams are located.  Each location has a variety of institution types in the near vicinity which allows great coverage.  We will also recruit some volunteers from various international locations to make sure we understand their users, context and work practices.

Our main goals in choosing users to work with are (also see overall research goals):

  • Comprehensive understanding of complex user population and system - how does managing content fit into the larger context of the work
  • Get users involved early in fleshing out the problem(s)
  • Understand the various workflows, needs & goals off Sakai users
  • Be able to identify patterns of use -- what's the 80% for various personas
  • Understand where variances occur so we can build in the flexibility at the appropriate places
  • Gain user empathy

Strategy / Plan:

  • Identify minimum number of users to meet above goals.  Using the matrix below to think through what the different user groups are, how various characteristics give them distinct needs and goal (ex. a user's institution has particular business practices they need to follow, the type and style of their class drives needs around sharing, collaboration, communication, etc.).  We'll make some educated guesses about how many users in what roles and where we need to schedule our initial interviews.
  • Recruit / organize teams of 2 in the identified locations.  Berkeley & Toronto are givens.  We'll likely need to recruit from UK, South Africa & Australia.  Each team will be responsible for identifying, recruiting and scheduling their contextual inquiries with the appropriate users.
  • Complete Contextual Inquiries (CI).  In teams of 2 the researchers will go to the place the user most often does their work.  In Higher Ed this gets tricky since users likely have several contexts in which they do their work.  We'll want to make sure we cover a variety of the likely contexts:  office, home, computer lab, coffee shops, etc.  A look at the contextual inquiries:
    • Always do the CI in teams of 2 if at all possible - one researcher should drive the conversation while the other takes detailed notes.  Switching back and forth is a good way to mix it up, make sure everyone gets a good mix of experience and have different perspectives in note taking and facilitating.
    • Contextual inquiries should be as conversational as possible.  We'll use guides to help us keep the conversation focused around content and our areas of interest but we don't want it to be like an interrogation where we just run down a list of questions.  It should be a natural conversation and we want users to show us their work as much as possible.  As with any user participation, it's important to introduce yourself, make the participant feel as comfortable as possible and let them know that they are not being tested and get a consent form signed.
    • 1 1/2 hours with the participant.  Start with global questions like "tell me about a typical day?", "what do you do first thing in the morning?", "what frustrates you about your work?"  "what is the favorite part of your work?".  This allows us to fit the rest of the conversation into context, early on identify some areas to probe around in more detail, and let the participant warm into the conversation.  As the conversation continues we want to get more and more detailed, asking the participant to show us what they do, or better yet if there is any real work to do, to do it.  Also be sure to ask for screen shots and/or copies of work  artifacts along the way (they can always forward to you later).  We'll shoot for 1 1/2 hours of inquiry time.  I usually like to stick to an hour to respect busy schedules but we have a lot to cover.
    • 2 hours of analyzing and digitizing what was learned -- immediately (or as immediately as possible).   Rather than just transcribe the notes, the researchers should analyze and make meaning of the notes and the experience with the participant while digitizing the notes.  I've found this collaborative processing to be invaluable in the past.
    • Share, discuss and access where we are at and what we know -- often (at least weekly).  With a distributed team it will be important for us to connect often to get synched up -- assess where we are, what we've been learning, what patterns are arising, what gaps are we seeing, etc.  We'll use one of our various on-line modes.
    • Face to face meeting to kick off modeling.  Once we've completed our initial round of contextual inquiries and feel we've begun seeing patterns, we hope to get together face to face for some period of time to work through some initial models of what we've learned -- personas, activity diagrams, use cases, scenarios, communication models, etc.

Other ways to slice it to minimize scope:

  • Supplement with some sort of user self report-- surveys, models and assumptions to react and add to, diaries, etc.
  • Focus on particular sets of user groups in an iteration.  This is tricky & won't give us the big picture.  For instance, if we focus only on Instructors first we'll miss the understanding of how Instructors pass a lot of work off to TAs and what the TAs role is in the processes.  We'll also miss the connections between how instructors share content with students and then students study, view & act on it.
  • Focus only on particular components.  This means we lose the goal of understanding a better global UI architecture for our systems based on organizing around content & activities rather than tools.  We could shorten the inquiries and focus on say uploading files.  I think this is also a mistake.  We won't understand how the components fit into the larger context of work.  We'll have to rely on what they do now rather than understanding their goals and what drives them to find better, innovative ways to support them.
  • Spread the inquiries out over a longer period of time (prior to modeling).  This can also be a bit dangerous.  Deep processing is going on for the researchers so the cost of switching tasks can be even higher than usual.  We also need to think about the outward appearance.  If the research drags on it can give onlookers the feeling often misunderstood perception that user research takes a long time.

Distinctions in process, structure, workflows, goals

Factors to consider:

  • Regional distinctions (particularly international)
  • Institution types (i.e. large research versus community college)
  • Departments (i.e sciences versus humanities)
  • Course type / structure (see below)
  • Job roles
  • Learning styles
  • Number & size of processes
  • Number of people involved in a process / collaboration involved
  • Time at institution / moving across institutions
  • Pedagogy styles
  • Computer interaction practices (i.e. keyboard only, screen reader user, etc.)
  • Course content type / subject matter
  • Also see the individual user matrices for more detail on some of these factors

Institutions identified

  • UC Berkeley
  • Stanford - Can the UX team at Stanford help out here?
  • Distance learning?
  • University of Toronto
  • York University
  • Cambridge
  • Unisa
  •  

Overall Matrix of Sakai Teaching & Learning Users

 

Region

Institution Type

Roles

Class type/structure

# to talk to in US

# to talk to in Canada

1

US / Canada

Public - Lg Research

Faculty - Lecturer / Instructor

Lg = >100 / Multiple sections

3

2

2

 

 

 

Small / seminar

 

 

3

 

 

 

Cross-listed

 

 

4

 

 

 

Multiple Instructors

 

 

5

 

 

 

No term

 

 

6

 

 

Faculty - Tenured / Tenured Track

Lg = >100 / Multiple sections

3

2

7

 

 


Small / seminar


 

8

 

 

 

Cross-listed

 

 

9

 

 

 

Multiple Instructors

 

 

10

 

 

 

No term

 

 

11

 

 

Faculty - Researcher

N/A

3

2

12

 

 

Department/School/Faculty Administrators


3

2

13

 

 

Instructional Designer

 

 

 

14

 

 

Teaching Assistant

Lg = >100 / Multiple sections

 

 

15

 

 

 

Small / seminar

 

 

16

 

 

 

Cross-listed

 

 

17

 

 

 

Multiple Instructors

 

 

18

 

 

 

No term

 

 

19

 

 

Student - undergrad

Various

3

2

20

 

 

Student - grad

Various

3

2

21

 

 

Student - phd

N/A

3

2

22

 

Public - Non-degreed (ex. Adult continuing education programs)

Faculty

N/A

Later phases if needed

 

23

 

 

Student

N/A

 

 

24

 

Public - State College 

Faculty - Lecturer / Instructor

 

Later phases if needed

 

25

 

 

Faculty - Tenured / Tenured track

 

 

 

26

 

 

Faculty - Researcher

 

 

 

27

 

 

Department/School/Faculty Administrators

 

 

 

28

 

 

Instructional Designer

 

 

 

29

 

 

Teaching Assistant

 

 

 

30

 


Student - undergrad

 


 

31

 


Student - grad

 


 

32

 

 

Student - phd?

 


 

33

 

Public - Community College

Faculty - Lecturer / Instructor

Small / seminar

3

2

33

 

 

 

Cross-listed


 

34

 

 

 

Multiple Instructors

 

 

35

 

 

 

No term 

 

 

36

 

 

Faculty - Tenured / Tenured track

Small / seminar

3

2

37

 



Cross-listed 

 

 

38

 

 

 

Multiple Instructors

 

 

39

 

 

 

No term

 

 

40

 


Faculty - Researcher

 

3

2

41

 

 

Department/School/Faculty Administrators

 

 

 

42

 

 

Instructional Designer

 

 

 

43

 

 

Teaching Assistant

 


2

44

 

 

Student

Various

3

2

45

 

Private

Faculty - Lecturer / Instructor

Lg = >100 / Multiple sections

3

N/A

46

 

 

 

Small / seminar 

 

 

47

 

 

 

Cross-listed 

 

 

48

 

 

 

Multiple Instructors

 

 

49

 

 

 

No term 

 

 

50

 

 

Faculty - Tenured / Tenured Track

Lg = >100 / Multiple sections

3

N/A

51

 

 

 

Small / seminar  

 

 

52

 

 

 

Cross-listed 

 

 

53

 

 

 

Multiple Instructors

 

 

54

 

 

 

No term  

 

 

55

 

 

Faculty - Researcher

 

3

N/A

56

 

 

Instructional Designer

 

 

 

57

 

 

Department/School/Faculty Administrators

 

 

 

58

 

 

Teaching Assistant 

Lg = >100 / Multiple sections

3

N/A

59

 

 


Small / seminar  

 

 

60

 

 

 

Cross-listed

 

 

61

 

 

 

Multiple Instructors

 

 

62

 

 

 

No term   

 

 

63

 

 

Student - Undergrad

Lg = >100 / Multiple sections

3

N/A

64

 

 


Small / seminar 

 

 

65

 

 

 

Cross-listed

 

 

66

 

 

 

Multiple Instructors

 

 

67

 

 

 

No term    

 

 

68

 

 

Student - Grad

Various

3

N/A

69

 


Student - phd

Various

3

N/A

70

 

Private - Distance Learning

Faculty - Lecturer / Instructor

Lg = >100 / Multiple sections

3

3

71

 

 

 

Small / seminar 

 

 

72

 

 

 

Cross-listed

 

 

73

 

 

 

Multiple Instructors

 

 

74

 

 

 

No term     

 

 

75

 

 

Department/School/Faculty Administrators

 

 

 

76

 

 

Instructional Designer

Various

3

3

77

 

 

Teaching Assistant  

Various

 

 

78

 

 

Student - Undergrad

Various

3

3

79

 

 

Student - Grad

Various

3

3

80

Australia

 

 

 

 

 

81

South Africa 

 

 

 

 

 

82

Europe Mainland

 

 

 

 

 

83

UK

 

 

 

 

 

84

 

 

 

 

 

 

85

 

 

 

 

 

 

86

 

 

 

 

 

 

87






 

 






 

 



 

 


 

 

South America





 

 

Egypt





 

 

China





 

 

Mexico

 

 

 

 

 

 


 

 

 

 

 

Working Area

Regions 

  • Australia
  • US (Do we need to distinguish between different regions here?)
  • Canada
  • South America
    • Columbia
  • South Africa
  • Egypt
  • China
  • Mexico
  • Europe -  what distinctions do we need to make here?  
    • Germany
    • Netherlands
    • Italy
    • Spain
    • Portugal
    • UK
    • Belgium
    • Ireland
    • Sweden
    • Turkey
    • Denmark

Types of Higher Education Institutions

How many of these are relevant in different regions?

Other types not so US-centric? 

  • Public
    • large research
    • extensions (not degreed programs)
    • state college (smaller)
    • community college (more mature student population?)
  • Private (what distinctions do we make here?)
    • Elite?
    • Small
    • Distance learning (also public?)

User Roles

  • Faculty
    • Lecturer
    • Tenured and/or tenured track
    • Other?
  • Teaching Assistants
  • Students
    • undergrad
    • grad
    • phd

Class Types

  • Large class
  • Small class
  • Class with multiple sections
  • Cross-listed class
  • Multiple instructors
  • Seminar
  • No term

Departments

  • Humanities
  • Languages
  • Sciences
  • Professional Schools 

Pedagogical styles

  • No labels