In user research, when do you choose a qualitative method over a quantitative one, and what can each kind of evidence tell you?
answer
- why and how vs how many
- discover first, then size
- small purposive vs large representative samples
- surveys only count listed answers
- triangulate the two
basics
~20 sQualitative methods such as interviews, observation and diary studies explain why and how people behave and uncover problems nobody knew to ask about. Quantitative methods such as surveys and product analytics measure how many and how much. Use qualitative to discover, quantitative to size.
solid answer
~50 sThe choice follows the question. **Qualitative** research — interviews, contextual inquiry, diary studies — answers *why* and *how*: it explains behaviour, reveals workarounds and finds problems you did not know existed, from a small, deliberately chosen sample. **Quantitative** research — surveys, product analytics — answers *how many* and *how much*: it measures the size of something you can already name, from a larger sample. On a B2B analytics dashboard, analytics might show that most accounts never open the report builder (what and how many); interviews reveal why — analysts export to spreadsheets because filters cannot express their fiscal quarters; a survey can then size how common that reason is. Strong research combines them, and treats a survey as a counting tool rather than a discovery tool, because it mostly captures answers the researcher already thought to offer.
go deeper
Recall the split: qualitative explains why and how from a small sample, quantitative measures how many from a larger one, and know one method of each.
Explain how to choose from the research question, why surveys mostly count answers you offered, and why small samples cannot give population percentages.
Design a mixed sequence — analytics, observation, a survey built from users' words, then a check on the change — and defend qualitative findings against the 'just anecdotes' objection.
Frame research investment as a portfolio: which decisions need explanation, which need measurement, and where triangulation is worth its cost.
## Two kinds of evidence **User research** is the systematic study of the people who use a product: what they do, what they need and why. Its methods fall into two families that answer different questions. | | Qualitative | Quantitative | |---|---|---| | Answers | Why? How? What is going on? | How many? How much? How often? | | Typical methods | Interviews, contextual inquiry, diary studies | Surveys, product analytics, experiments | | Sample | Small, chosen on purpose | Larger, ideally representative | | Output | Themes, explanations, quotes, observed workarounds | Counts, proportions, trends, comparisons | | Main weakness | Cannot say how common something is | Cannot explain why, or find what it did not ask about | Neither is better. They are tools for different jobs, and choosing the wrong one produces confident answers to the wrong question. ## When qualitative comes first Use qualitative methods when you **do not yet know what the problem is**: - A new product area, a new customer segment or a new workflow. - A metric has moved and nobody can explain it. - The team is arguing from assumptions about how users work. - You need the language users themselves use, to write good survey options later. On a B2B analytics dashboard, product analytics show that most accounts never open the report builder. That number cannot say why. Watching and talking to eight analysts can: some export to spreadsheets because the filters cannot express their company's fiscal quarters, others never discovered the builder, and a few are not allowed to share dashboards outside their team. ## When quantitative comes first Use quantitative methods when you **can already name the thing and need its size**: - How many accounts hit the fiscal-quarter limitation? - Which of five known pain points is most common in each customer segment? - Did a change move the behaviour it was meant to move? Quantitative results also tell qualitative research **where to look**: the drop-off step in a funnel is where the interviews should focus. ## Where surveys fit, and their traps A **survey** is a quantitative tool that relies on self-report. It works for attitudes, satisfaction and the frequency of known issues. Its traps: 1. **It mostly captures answers you offered.** A multiple-choice question about why users stopped using a feature rarely reveals a reason nobody listed; an open text box helps but gets short, uneven answers. 2. **Self-reported behaviour is unreliable.** People misremember how often they do things and predict their future behaviour poorly. 3. **Wording and order bias answers**: leading phrasing, double-barrelled questions, and scales without a neutral point all skew results. 4. **Who responds is not random**: the most engaged or most annoyed users answer more. ## Combining them: a worked sequence 1. **Analytics** show where the report builder loses users. 2. **Contextual inquiry and interviews** explain why, and collect the users' own words. 3. **A survey** uses those words as options and sizes each reason across segments. 4. **Product changes** target the largest reasons, and analytics or a controlled experiment check the effect. This is **triangulation**: confidence comes from different methods pointing the same way. ## Sample sizes, and why they differ Qualitative studies use small, **purposive** samples: participants chosen because they have the behaviour in question. A common practice is to continue until new sessions stop adding new themes, often called **saturation**, which for one well-defined segment frequently arrives somewhere between five and twelve interviews; that range is a rule of thumb, not a standard. Quantitative studies need samples large enough for an estimate to be precise, drawn so that respondents resemble the population; a thousand responses from only the most engaged customers are still a biased thousand. The two are therefore reported differently: qualitative findings as themes with participant counts, quantitative results as estimates with their uncertainty. ## Common mistakes - Reporting "58 percent of users" from twelve interviews: a small purposive sample cannot estimate a population share. - Running a survey to discover problems, then treating the listed options as the full picture. - Dismissing qualitative findings as anecdotes, when their job is explanation, not measurement. - Treating a large number of responses as proof of representativeness.
- Can you count anything in qualitative research?Yes, as long as the count describes the sample rather than the population. Saying seven of ten analysts exported to spreadsheets shows how broad a theme was among the people you spoke to, which helps prioritise. What you cannot do is turn that into a percentage of all users; that needs a quantitative method with a suitable sample.
- Why use participants' own words when writing survey options?Because options written in the team's vocabulary may not match how users think about the problem, and respondents then pick the nearest wrong option. Qualitative research collects the terms users actually use, such as 'my fiscal quarters' rather than 'custom date ranges', so the survey measures the real reasons instead of the team's guesses.
saying these in an interview costs you the question
- Qualitative findings are just anecdotes until a survey confirms them.
- A survey with many responses reveals problems nobody had thought of.
- Twelve interviews can tell you what percentage of users share a need.
- Quantitative data is always more reliable than qualitative data.
- Asking users whether they would use a feature predicts adoption well.