Body
Anne Deutsch, PhD, RN, a research scientist in the Center for Rehabilitation Outcomes Research (CROR), discusses cognitive interviewing and its role in a project within CROR’s Rehabilitation Research and Training Center on Home and Community-based Services (RRTC HCBS).
As part of the Rehabilitation Research and Training Center on Home and Community-based Services, you are developing surveys for HCBS users. Can you tell us about this work?
We are developing survey questions that ask HCBS users about nine aspects of their lives and whether services and supports support them in these areas. The nine areas are community engagement; choice and control over living arrangements; choice and control over money; choice and control over healthcare and health habits; choice and control over personal expression; choice and control over food and nutrition; choice and control over how time is spent; choice and control over meaningful relationships and dignity of risk.
We published a paper that described how these topics were selected.
Question: Can you explain cognitive interviewing?
Cognitive interviewing focuses on the thought processes respondents use when answering questions. Rather than treating answers as simple data points, the method examines how an answer was produced — comprehension, memory retrieval, judgment and response selection.
The approach is grounded in cognitive psychology, specifically the idea that answering a question involves four stages:
- Comprehension – Understanding what the question is asking
- Retrieval – Recalling relevant information from memory
- Judgment – Evaluating and integrating recalled information
- Response – Mapping the judgment onto the provided answer options
Cognitive interviewing helps identify breakdowns at any of these stages.
Question: Why is cognitive interviewing important for developing survey items?
It can help improve the clarity and validity of questions and reveal whether respondents interpret questions as intended. Even small wording issues can lead to misinterpretation, which threatens the validity of the items.
For example, a question intended to measure community engagement may be interpreted by some respondents as only in-person interactions, while others may include virtual and electronic interactions.
Cognitive interviews can also identify hidden sources of measurement error including ambiguous terms, double-barreled questions, inappropriate recall periods or mismatched response options. Cognitive interviewing helps uncovers these errors before data collection begins.
Cognitive interviews can also enhance inclusiveness. Different populations (age, education, culture, health status) may interpret the same question differently. Cognitive interviewing helps ensure that items work across groups of people with different backgrounds and characteristics, which should reduce bias.
Cognitive interview help improving data quality and interpretability. When respondents understand questions consistently and can answer accurately, responses are more reliable, comparisons across individuals or groups are more meaningful and fewer post hoc corrections or exclusions are needed.
Finally, cognitive interviews will save time and resources in the long-term. Although cognitive interviewing requires upfront effort, it prevents costly problems later, such as invalid study conclusions, poor psychometric performance and the need for survey redesign after data collection.
How is cognitive interviewing conducted?
Cognitive interviews are typically conducted one-on-one and use verbal probing techniques. Two core approaches are often combined:
1. Think-Aloud Method
Participants are asked to verbalize their thoughts as they answer each question.
- Example: “Please tell me everything that goes through your mind as you answer this.”
2. Verbal Probing
The interviewer asks targeted follow-up questions to explore specific issues, such as:
- Comprehension probes: “What does the term moderate pain mean to you?”
- Recall probes: “How did you remember that information?”
- Confidence probes: “How sure are you about that answer?”
- Response mapping probes: “Did any answer choices not quite fit your situation?”
Interviews are usually recorded, transcribed, and analyzed to identify recurring problems.
Can you summarize in two sentences why cognitive interviews are an important step in survey development?
Cognitive interviewing helps researchers see questions from the respondent’s perspective. By uncovering misunderstandings, assumptions, and cognitive load, it ensures that subsequent collected data truly reflect what researchers intend to measure.
Are you planning to publish this research?
Yes, we are refining the paper that describes this work and the findings.