skip to content

Analysts keep abandoning a B2B analytics dashboard's report builder; how would you plan the user research and choose between interviews, contextual inquiry and a diary study?

level: middleimportance: should knowfreq 40%

answer

  1. start from the decision it informs
  2. say vs do
  3. behaviour over time
  4. who, how many, how recruited
  5. analysis and deliverable decided up front

basics

~20 s

Start the plan from the research question and the decision it informs, then pick the method that answers it: interviews for recalled experience, contextual inquiry to observe real work and workarounds, a diary study for behaviour spread over weeks.

solid answer

~50 s

A **research plan** begins with the **research question** and the **decision** it will inform — here, "why do analysts abandon the report builder, and what should we change first?" — then sets objectives, method, participants, recruiting, sample size, logistics, the discussion guide, the analysis approach, the deliverable and the timeline, and gets stakeholders to agree before fieldwork. The method follows the question. **Interviews** are fast and good for recalled experience and attitudes across roles. **Contextual inquiry** — watching analysts build a real report in their own setting and asking as they go — reveals what people do rather than what they say, including spreadsheet workarounds. A **diary study** captures behaviour spread over time, such as a month-end reporting cycle. For abandonment, analytics would first show where users drop off; contextual inquiry would then explain why, with a diary study added if the problem appears only at certain points in the cycle.

go deeper

for a junior

Recall what a research plan contains and what interviews, contextual inquiry and diary studies are each for.

for a middle

Choose the method from the question — say versus do, a moment versus a period, breadth versus depth — and write a plan that states the decision it informs.

for a senior

Sequence methods for a real problem, starting from analytics, and handle B2B constraints such as site access, confidential data and participant availability.

for a principal

Judge which product decisions deserve a planned study at all, and make agreeing the question with stakeholders a standing practice rather than an afterthought.

## Start with the question, not the method A **research plan** is a short document that makes a study's purpose and design explicit before anyone talks to users. Its first line is the **research question** and the **decision** the answer will inform. For a B2B analytics dashboard whose report builder analysts keep abandoning, that might be: *Why do analysts abandon the report builder, and which cause should we address first?* Everything else in the plan follows from it. ## What goes into the plan 1. **Background**: what is already known — analytics, support tickets, earlier studies. 2. **Research question and objectives**, including what is out of scope. 3. **Method**, and why it answers this question. 4. **Participants**: the behaviours and roles that qualify, such as analysts who started a report in the last month; and the segments to cover, such as new and long-standing accounts. 5. **Recruiting and sample**: channels, screener, target numbers, incentives. 6. **Logistics**: remote or on site, session length, consent and recording, data handling. 7. **Discussion guide or task outline.** 8. **Analysis approach and deliverable**: how findings will be synthesised and shared. 9. **Timeline and stakeholders**: who observes, who reviews, when decisions are made. Agreeing the plan up front prevents the most common failure: a study that answers a question nobody needed answered. ## Choosing between the three methods | Method | What it is | Best for | Limits | |---|---|---|---| | **Interview** | A one-to-one conversation about experience | Recalled experience, goals, attitudes, covering many roles quickly | Relies on memory and self-report | | **Contextual inquiry** | Observing a person doing real work in their own setting, asking questions as they go | What people actually do: workarounds, tools, interruptions, data sources | Time-consuming; access can be hard in B2B | | **Diary study** | Participants record entries about an activity over days or weeks | Behaviour over time, infrequent events, changes across a cycle | Participant effort, drop-out, uneven entries | Key distinctions: - **Say versus do**: interviews capture what people say; contextual inquiry shows what they do. When the two may differ — and with established workflows they often do — observation is the stronger evidence. - **A moment versus a period**: interviews and observation capture a session; a diary study captures a pattern over time. - **Breadth versus depth**: interviews reach more roles quickly; contextual inquiry goes deeper with fewer people. ## Applying it to the report builder - **Analytics first** show *where* analysts drop off: at the filter step, at sharing, or before saving. - **Contextual inquiry** with a handful of analysts building a real report explains *why*: perhaps they switch to a spreadsheet when the filters cannot express their fiscal quarters, or when a colleague needs the data in a different shape. - **Interviews** with managers who consume the reports add the other side of the workflow. - **A diary study** is worth adding if abandonment clusters around month-end or quarterly reporting, which a single session will miss. ## Sizing the effort A plan also makes the cost visible. Interviews with one segment often take a week or two including recruiting; contextual inquiry takes longer per participant because sessions must be scheduled around real work, and sometimes travel; a diary study runs for its whole period plus set-up and wrap-up conversations. These are planning estimates that vary widely between organisations, not fixed durations. Stating them lets stakeholders weigh the study against the decision's deadline, and choose a smaller study that lands in time over a thorough one that arrives after the decision has been made. ## Common planning mistakes - Choosing the method the team is comfortable with, then fitting the question to it. - Leaving the analysis approach undecided, so notes pile up with no plan to make sense of them. - Recruiting whoever is easiest to reach instead of the behaviours the question needs. - Skipping stakeholder agreement, so findings arrive after the decision was made. A plan is the cheapest part of a study, and the part that most often decides whether the rest was worth doing.

  • How do you run contextual inquiry when a B2B customer will not let you on site?
    Run it remotely: the participant shares their screen while doing a real task, ideally with their own data if they are allowed to show it, and you ask questions as they work. You lose the physical environment — interruptions, second monitors, paper notes — so ask about those explicitly. Agree up front what data may be shown and recorded.
  • What makes a diary study succeed rather than fizzle out?
    Short, focused prompts tied to the moment of interest, such as each time a report is built; reminders; an easy entry format, including photos or screenshots; an incentive paid in stages rather than at the end; and a kick-off and wrap-up interview so entries can be explained. Without these, entries thin out after the first days.

saying these in an interview costs you the question

  • Pick the method first, then decide what to ask.
  • Interviews reveal workarounds as reliably as watching people work.
  • A single session captures behaviour that happens once a month.
  • The analysis approach can be decided after all the sessions are done.
  • A research plan is paperwork that slows the team down.