Short answer

Incorporate methods that allow for the detection and correction of response bias to ensure the validity of market research findings.

Field: Innovation & Markets
Source: Journal of Marketing Research (2015)
Method: Bayesian Item Response Theory (IRT) modeling with a dual-questioning approach.
Sample: 1,408 participants
Evidence: Strong effect

Employing a dual-questioning technique that contrasts direct and privacy-protected responses can quantify and account for under- and overreporting in survey data. This innovation & markets research insight is drawn from a 2015 study published in Journal of Marketing Research. Using Bayesian item response theory (irt) modeling with a dual-questioning approach. with 1,408 participants, researchers explored how this design variable affects real-world outcomes. The key design takeaway: Incorporate methods that allow for the detection and correction of response bias to ensure the validity of market research findings.

Study

Innovation & MarketsHigh ImpactStrong effect

Dual-Questioning Technique Uncovers Hidden Response Bias in Market Research

Employing a dual-questioning technique that contrasts direct and privacy-protected responses can quantify and account for under- and overreporting in survey data.

Journal of Marketing Research · 2015

Key Findings

01Respondents exhibit aversion to decreased privacy when transitioning from direct to randomized response questioning.
02Randomized response techniques may be less effective if respondents have already provided biased responses to earlier direct questions.
03The proposed Bayesian IRT model can quantify under- and overreporting, accounting for individual differences in bias direction.

Application

Design takeaway

Incorporate methods that allow for the detection and correction of response bias to ensure the validity of market research findings.

How to apply

When designing surveys for sensitive topics or behaviors, consider including a second set of questions using a randomized response technique to cross-validate direct answers.

Project actions

01When designing surveys for your project, think about how to ask sensitive questions to get honest answers.
02Consider if a follow-up question or a different method could help verify initial responses.

Method & Evidence

AimHow can a dual-questioning technique be used to quantify under- and overreporting in survey responses, particularly in behavioral domains?

MethodBayesian Item Response Theory (IRT) modeling with a dual-questioning approach.

ProcedureRespondents answered questions directly, followed by answering similar questions using a randomized response technique that offers item-level privacy. Differences between these responses were analyzed using a Bayesian IRT model to quantify reporting deviations.

Sample1,408 participants

ContextMarket research, consumer behavior surveys, behavioral domains (e.g., alcohol consumption).

Variables

IVQuestioning technique (direct vs. randomized response).

DVReported behavior/attitude, deviation from truth (quantified under/overreporting).

CVParticipant characteristics, specific behavior being measured, order of questioning.

Strengths & Limitations

Strengths

+Introduces a novel statistical model for quantifying response bias.
+Provides empirical evidence from a real-world application.

Limitations

Implementing randomized response techniques can be complex and may require specialized software or careful instruction to participants.

Reliability & validity

The Bayesian IRT model aims to improve the validity of survey data by accounting for response bias. Reliability would depend on the consistency of responses across similar questions and the stability of the model's parameters.

Think critically

How might the 'response-mode inertia effects' mentioned in the abstract influence the results, and what are the implications for designing survey sequences?

Design Principles

"Data validity is enhanced by employing methodologies that account for inherent human biases in self-reporting."

This methodology allows for a more accurate understanding of consumer behavior and attitudes by identifying and correcting for social desirability bias and other reporting inaccuracies. Accurate data is crucial for effective product development, marketing strategies, and competitive analysis.

What This Means for Your Design

This research shows a clever way to find out if people are telling the truth in surveys by asking them the same question twice, once directly and once in a way that protects their privacy. It helps researchers get more accurate information.

How to use in your project

1.This study provides a robust methodology for addressing response bias, which can be cited when discussing the limitations of self-reported data in your own research project and how you mitigated or accounted for it.

Add to My Project

Quick Cite

Paragraph starter

The study by de Jong, Fox, and Steenkamp (2015) highlights the challenge of response bias in surveys, particularly in behavioral domains. Their research introduces a dual-questioning technique, contrasting direct and randomized response methods, which, when analyzed with Bayesian IRT modeling, can quantify under- and overreporting. This approach is valuable for ensuring data accuracy in design research by accounting for social desirability and privacy concerns.

Source

Journal of Marketing Research

Quantifying Under- and Overreporting in Surveys through a Dual-Questioning-Technique Design

journal · 2015

View source

Questions About This Research

What does the research say about dual-questioning technique uncovers hidden response bias in market research?: Incorporate methods that allow for the detection and correction of response bias to ensure the validity of market research findings. Evidence: Journal of Marketing Research (2015).
Why does "Dual-Questioning Technique Uncovers Hidden Response Bias in Market Research" matter for design?: This methodology allows for a more accurate understanding of consumer behavior and attitudes by identifying and correcting for social desirability bias and other reporting inaccuracies. Accurate data is crucial for effective product development, marketing strategies, and competitive analysis.
How can designers apply this research?: Incorporate methods that allow for the detection and correction of response bias to ensure the validity of market research findings.
What were the main findings?: Respondents exhibit aversion to decreased privacy when transitioning from direct to randomized response questioning.. Randomized response techniques may be less effective if respondents have already provided biased responses to earlier direct questions.. The proposed Bayesian IRT model can quantify under- and overreporting, accounting for individual differences in bias direction.
What research method was used?: Bayesian Item Response Theory (IRT) modeling with a dual-questioning approach. with 1,408 participants.
How strong is the evidence?: Evidence strength is rated Strong effect, based on a 2015 journal from Journal of Marketing Research.
What should I do differently in my next project?: When designing surveys for sensitive topics or behaviors, consider including a second set of questions using a randomized response technique to cross-validate direct answers.
What are the limitations?: The effectiveness of randomized response can be influenced by prior direct questioning, and privacy loss itself can introduce bias.