Short answer

Design systems and processes that acknowledge and leverage the expertise of experienced users, especially in dynamic and unpredictable operational contexts.

Field: Innovation & Design
Source: Lund University Publications Student Papers (Lund University) (2017)
Method: Qualitative case study
Evidence: Strong effect

In highly complex and evolving systems like aviation, strict adherence to pre-defined rules can hinder effective decision-making when unexpected situations arise, necessitating reliance on pilot experience and knowledge. This innovation & design research insight is drawn from a 2017 study published in Lund University Publications Student Papers (Lund University). Using Qualitative case study, researchers explored how this design variable affects real-world outcomes. The key design takeaway: Design systems and processes that acknowledge and leverage the expertise of experienced users, especially in dynamic and unpredictable operational contexts.

Study

Innovation & DesignHigh ImpactStrong effect

Experience Trumps Rules in Complex Aviation Scenarios

Lund University Publications Student Papers (Lund University) · 2017

Key Findings

01Pilots often encounter situations not covered by existing rules.
02In unexpected situations, pilots rely heavily on their knowledge and experience, not just rules.
03A gap can exist between how rules are written and how they are applied in practice.
04Pilots must use more than just rules and procedures to make decisions and solve problems in complex situations.

Application

Design takeaway

Design systems and processes that acknowledge and leverage the expertise of experienced users, especially in dynamic and unpredictable operational contexts.

How to apply

When designing complex systems, conduct in-depth qualitative research with experienced users to understand how they navigate exceptions and ambiguities, and incorporate these insights into system design and training.

Project actions

01When designing a product, think about how real users will actually use it, especially in difficult or unusual circumstances.
02Consider how your design might need to adapt if a user encounters something you didn't anticipate.

Method & Evidence

AimTo understand how pilots integrate rules and procedures into their decision-making processes during challenging and unexpected situations in aviation.

MethodQualitative case study

ProcedureThe study explored how pilots make decisions in difficult and challenging situations by examining their integration of rules and procedures, using a qualitative case study methodology.

ContextAviation industry

Variables

IVNature of the situation (expected vs. unexpected)

DVPilot decision-making process (reliance on rules vs. experience)

CVType of aircraft, pilot experience level, specific flight phase

Strengths & Limitations

Strengths

+Provides a qualitative, in-depth understanding of a complex human-system interaction.
+Focuses on real-world application in a safety-critical industry.

Limitations

The findings might not apply to simpler systems or contexts where strict adherence to rules is always the safest approach.

Reliability & validity

The qualitative nature of case studies can offer rich insights but may have limitations in generalizability. Reliability could be enhanced through triangulation of data sources (e.g., interviews, observations, documentation).

Think critically

To what extent should designers aim to create systems that are entirely rule-bound versus those that empower user discretion, and how can this balance be achieved?

Design Principles

"Design for adaptability: Systems should be flexible enough to accommodate unforeseen circumstances and allow for expert judgment."

This insight is crucial for designers and engineers developing systems with human operators in safety-critical domains. It highlights the limitations of purely rule-based design and emphasizes the need to account for human expertise and adaptability.

What This Means for Your Design

In jobs like flying planes, where things can get tricky and unexpected, just following a rulebook isn't always enough. People with lots of experience can make better decisions by using their own knowledge when rules don't quite fit the situation.

How to use in your project

1.Reference this study when discussing the limitations of purely prescriptive design and the importance of user experience in your design process.

Add to My Project

Quick Cite

Paragraph starter

This research highlights that in complex operational environments, such as aviation, strict adherence to pre-defined rules can be insufficient when unexpected situations arise. Pilots often rely on their accumulated knowledge and experience to navigate these ambiguities, suggesting that design processes should actively incorporate user expertise and allow for adaptive decision-making beyond rigid procedural frameworks.

Source

Lund University Publications Student Papers (Lund University)

Rules - Sometimes less is more

journal · 2017

View source

Questions About This Research

What does the research say about experience trumps rules in complex aviation scenarios?: Design systems and processes that acknowledge and leverage the expertise of experienced users, especially in dynamic and unpredictable operational contexts. Evidence: Lund University Publications Student Papers (Lund University) (2017).
Why does "Experience Trumps Rules in Complex Aviation Scenarios" matter for design?: This insight is crucial for designers and engineers developing systems with human operators in safety-critical domains. It highlights the limitations of purely rule-based design and emphasizes the need to account for human expertise and adaptability.
How can designers apply this research?: Design systems and processes that acknowledge and leverage the expertise of experienced users, especially in dynamic and unpredictable operational contexts.
What were the main findings?: Pilots often encounter situations not covered by existing rules.. In unexpected situations, pilots rely heavily on their knowledge and experience, not just rules.. A gap can exist between how rules are written and how they are applied in practice.. Pilots must use more than just rules and procedures to make decisions and solve problems in complex situations.
What research method was used?: Qualitative case study.
How strong is the evidence?: Evidence strength is rated Strong effect, based on a 2017 journal from Lund University Publications Student Papers (Lund University).
What should I do differently in my next project?: When designing complex systems, conduct in-depth qualitative research with experienced users to understand how they navigate exceptions and ambiguities, and incorporate these insights into system design and training.
What are the limitations?: The study's findings may be specific to the aviation context and the particular types of unexpected situations encountered by pilots.