Short answer

Prioritize observing and measuring actual user behaviour in real-world contexts over relying solely on self-reported data to ensure design solutions are effective and desirable.

Field: User-Centred Design
Source: Annals of Tourism Research (2025)
Method: Comparative analysis of research methodologies (cross-sectional surveys, survey experiments, field experiments) and practical guidance on conducting field experiments.
Evidence: Strong effect

Measuring actual behaviour in real-world settings through field experiments provides more reliable insights than self-reported data from surveys or survey experiments. This user-centred design research insight is drawn from a 2025 study published in Annals of Tourism Research. Using Comparative analysis of research methodologies (cross-sectional surveys, survey experiments, field experiments) and practical guidance on conducting field experiments., researchers explored how this design variable affects real-world outcomes. The key design takeaway: Prioritize observing and measuring actual user behaviour in real-world contexts over relying solely on self-reported data to ensure design solutions are effective and desirable.

Study

User-Centred DesignNew This WeekStrong effect

Field Experiments Reveal Discrepancies Between Stated Preferences and Actual Behaviour

Measuring actual behaviour in real-world settings through field experiments provides more reliable insights than self-reported data from surveys or survey experiments.

Annals of Tourism Research · 2025

Key Findings

01Survey-based methods (cross-sectional surveys and survey experiments) often fail to capture actual behaviour, leading to potential misinterpretations.
02Field experiments, by observing behaviour in natural settings, provide more accurate and actionable insights than laboratory or survey-based studies.
03There can be significant empirical evidence of discrepancies between conclusions drawn from survey data and those from field studies.

Application

Design takeaway

Prioritize observing and measuring actual user behaviour in real-world contexts over relying solely on self-reported data to ensure design solutions are effective and desirable.

How to apply

When developing a new product or service, conduct a small-scale field study or pilot test to observe how target users interact with a prototype or early version in their natural environment.

Project actions

01Consider how you can observe real user behaviour rather than just asking for opinions.
02Think about the ethical implications of observing people in their natural environment.

Method & Evidence

AimTo investigate the discrepancies between user behaviour in survey-based studies and real-world behaviour, and to establish the utility of field experiments for generating actionable design insights.

MethodComparative analysis of research methodologies (cross-sectional surveys, survey experiments, field experiments) and practical guidance on conducting field experiments.

ProcedureThe study systematically compares the strengths and limitations of different research approaches, focusing on how they capture user behaviour. It provides a step-by-step guide for designing and implementing field experiments and quasi-experimental field studies, including ethical considerations.

ContextTourism and hospitality research, but applicable to any design domain where understanding user behaviour is critical.

Variables

IVResearch methodology (cross-sectional survey, survey experiment, field experiment)

DVObserved user behaviour, stated user preference

CVParticipant demographics, specific product/service being studied, environmental factors (in field experiments)

Strengths & Limitations

Strengths

+Provides a systematic comparison of research methods.
+Offers practical guidance for conducting field experiments.

Limitations

Field experiments can be difficult to control, and ethical approval might be needed.

Reliability & validity

Field experiments generally offer higher ecological validity (realism) than survey experiments, but may have lower internal validity due to less control over extraneous variables. Reliability can be a concern if the observed behaviour is highly context-dependent.

Think critically

How might the context of a field experiment influence behaviour, potentially creating a new form of artificiality?

Design Principles

"Reveal user needs through observed behaviour, not just stated preferences."

Designers often rely on user feedback gathered through surveys or controlled experiments. However, stated intentions can differ significantly from actual actions. Field experiments allow for the observation of genuine behaviour, leading to more accurate understanding of user needs and more effective design solutions.

What This Means for Your Design

Surveys ask people what they like, but field experiments watch what they actually do. What people do is usually more important for designers than what they say.

How to use in your project

1.Use this research to justify the methodology chosen for your design project, especially if you are moving beyond simple surveys to observe user interaction.
2.Cite this paper when discussing the limitations of survey data and the benefits of real-world observation for gathering user insights.

Add to My Project

Quick Cite

Paragraph starter

This research underscores the critical distinction between stated user preferences, often gathered through surveys, and actual user behaviour. By advocating for field experiments, the study demonstrates that observing users in their natural environments yields more reliable and actionable insights, crucial for developing effective design solutions that truly resonate with user needs and habits.

Source

Annals of Tourism Research

Field experiments: Overcoming the limitations of survey experiments for actionable behavioural insights

journal · 2025

View source

Questions About This Research

What does the research say about field experiments reveal discrepancies between stated preferences and actual behaviour?: Prioritize observing and measuring actual user behaviour in real-world contexts over relying solely on self-reported data to ensure design solutions are effective and desirable. Evidence: Annals of Tourism Research (2025).
Why does "Field Experiments Reveal Discrepancies Between Stated Preferences and Actual Behaviour" matter for design?: Designers often rely on user feedback gathered through surveys or controlled experiments. However, stated intentions can differ significantly from actual actions. Field experiments allow for the observation of genuine behaviour, leading to more accurate understanding of user needs and more effective design solutions.
How can designers apply this research?: Prioritize observing and measuring actual user behaviour in real-world contexts over relying solely on self-reported data to ensure design solutions are effective and desirable.
What were the main findings?: Survey-based methods (cross-sectional surveys and survey experiments) often fail to capture actual behaviour, leading to potential misinterpretations.. Field experiments, by observing behaviour in natural settings, provide more accurate and actionable insights than laboratory or survey-based studies.. There can be significant empirical evidence of discrepancies between conclusions drawn from survey data and those from field studies.
What research method was used?: Comparative analysis of research methodologies (cross-sectional surveys, survey experiments, field experiments) and practical guidance on conducting field experiments..
How strong is the evidence?: Evidence strength is rated Strong effect, based on a 2025 journal from Annals of Tourism Research.
What should I do differently in my next project?: When developing a new product or service, conduct a small-scale field study or pilot test to observe how target users interact with a prototype or early version in their natural environment.
What are the limitations?: Field experiments can be more complex and costly to implement than survey experiments, and may involve unique ethical considerations.