Project: Tandem Mental Model Research

Facts

Project timeframe: May-September 2022

Team: User Research Team at CAI, Samuel Wood & Dominic Bonanni

My role: User Researcher

Users:  Instructors who teach semester-long team-based courses

Stakeholders: UX designers, research directors, QA leads, PMs, & others. 

Tools: Structured interviews, mental model frameworks, project management boards, screener surveys

Software: Figma, Dovetail, Miro, Qualtrics

Skills: Qualitative data analysis, human-centered research, project management, user recruitment, strategic research planning, survey design and others

Fig. 1: Part of our completed Mental Diagram

Fig. 1: Part of the completed Mental Diagram

Problem

Tandem is an educational technology software platform that aids instructors in forming student teams that are equitable and that properly balance students according to the needs of the instructor. The Instructor-Led Onboarding (ILO) process is specifically designed to aid instructors by providing a self-sufficient means of setting up a course in Tandem. We need to understand how instructors currently think about setting up courses with student teams, in order to meet their needs with the ILO process. I engaged in UX research over the summer of 2022 to figure out how instructors tend to think about the course setup process.

Goals

Fig 2: An example of a mental model from Indi Young’s website

Understand mental models- Uncover and visualize the mental models—the thought-processes, behaviors, motivations, emotions and philosophies—instructors employ as they set up and teach team-based courses.

Create archetypes/personas- Take these mental models and devise user archetypes that represent the differences between types of instructors who create team-based courses, and how these different user types may interact with Tandem.

Research Questions

  • What choices do instructors make as part of their setup and teaching process for courses with student teamwork? 

  • What mental spaces do instructors occupy when making these choices?

  • What tools, resources, and personnel support instructors during this process?

  • What pain points do instructors experience during the setup process? 

  • How do mental models for course setup differ between instructors? 

  • Background:

    I always start a research project by osmosis. I take in as much context as possible, soak that all up, and attempt to separate what I need to know from what I don’t need to know. I began this particular project by reading a book, specifically, Indi Young’s Mental Models: Aligning Design Strategy with Human Behavior. Since we wanted to understand a wide range of thought processes for instructors engaged in course setup, I focused on Young’s insights regarding how to organize and categorize data.

  • Data:

    Recruitment and sampling are processes that always have lots of moving parts, so I stay organized and prepared in a variety of ways when approaching both. When selecting a sample for this study, I started with a list of instructors who had previously reached out to us about using Tandem. I arranged these in an Excel sheet and funneled these down based on last time contacted, degree of interest, and other factors. Once I have a sample that makes sense, I try to already have a short screener survey that I can use to reach out and start connecting with potential users.

  • Categorization: 

    I like to approach user data as soon after collection as possible, in order to remember the full context and approach the data with the same mindset that I had when collecting that data. With that in mind, I combed through each interview transcript in Dovetail and tagged relevant information as soon as I had completed each interview. I try to tag specific topics that users talk about, as well as how users think about those topics and how they act or react when making decisions about those topics. I use this multilayered tagging to make my process easier later on, in particular how I organize and display my data. I also tend to use color-coding to help distinguish different topics from each other.

Process

  • Affinitization:

    When arranging my data pieces (consisting of transcript highlights and concepts expressed by users), I started off by using the treemap view in Dovetail to get the big picture of which areas users tended to focus on and what sentiments were expressed. Getting this big picture view helps me really understand the structure of the data, which then allows me to fill in smaller pieces later on. Once I have this structure figured out, I comb through each topic and try to tell a story that can lead to a measurable improvement.

  • Presentation:

    All of this process makes sense to me, but I have to explain this to others. I find that good stories leave space  for the audience to fill in the blanks with their imagination. When I present my work, I try to encourage that participation and interest from my stakeholders. When presenting this particular project, I reached out to stakeholders by offering them a workshop where they could engage in gap analysis along with the User Research team. This enabled our stakeholders to identify matches and mismatches between Tandem’s abilities and the mindsets of our users. I find that letting our team fill in the blanks provided myself and the rest of the team with a chance to synthesize our insights in a collaborative and effective manner.

Process Continued

  • Instructors fit into two main archetypes: Teamwork Optimizer and Teamwork Pragmatist. Teamwork Optimizers tended to create opportunities for project-based learning and valued teamwork as an educational goal. Teamwork Pragmatists tended to have less applied coursework focuses and thought of teamwork as more of a helpful additional skill than a primary focus. 

  • Commonalities between instructor archetypes include a high degree of communication with other instructors when initially setting up courses, engaging in experimenting with course format & team size, seeking demographic info from students, creating clear expectations for student teams, and rearranging teams when necessary.

Findings

Takeaways

Fig 3: Instructor archetypes

  • Instructor differences were prototyped with the Teamwork Optimizer and Teamwork Pragmatist archetypes, but instructors often fell somewhere on a spectrum between the archetypes. User archetypes can help greatly, but they don’t necessarily tell the entire story. 

  • Mental models provide a very helpful way to see where a software solution is helpful and where it is lacking. Aligning software features with user behaviors makes the gaps and the solution spaces crystal clear.

  • Tandem stakeholders were able to take instructor mental models and user archetypes into account when scoping and creating 12 improvements to Tandem Instructor-Led Onboarding.

  • Tandem staff and the R&D team gained insight into the thought processes of instructors during the crucial initial stage of the setup process. 

  • UXR and other Tandem stakeholders now have mental models and instructor archetypes that can serve as a springboard for future iterations of Tandem research. 

Impact

Appendix

Fig. 2: Indi Young’s Mental Models, a primary resource for this study
Fig. 3: Some of the screener questionnaire questions on Qualtrics

Fig. 4: Indi Young’s Mental Models, a primary resource for this study

Fig. 5: Some of the screener questionnaire questions on Qualtrics

Fig. 6: Content map of Tandem-instructor interactions