Short answer

Design interactive visual analytics systems that actively support and capture the collaborative reasoning process between domain experts and tool specialists to uncover deeper insights and improve system design.

Field: Human Factors
Source: Summit (Simon Fraser University) (2011)
Method: Qualitative research, Case study, Method development and evaluation
Evidence: Strong effect

Collaborative sensemaking between subject matter experts and tool experts, facilitated by visual analytics, significantly improves the understanding of complex aviation safety data. This human factors research insight is drawn from a 2011 study published in Summit (Simon Fraser University). Using Qualitative research, case study, method development and evaluation, researchers explored how this design variable affects real-world outcomes. The key design takeaway: Design interactive visual analytics systems that actively support and capture the collaborative reasoning process between domain experts and tool specialists to uncover deeper insights and improve system design.

Study

Human FactorsHigh ImpactStrong effect

Paired Analysis: Enhancing Aviation Safety Sensemaking Through Expert Collaboration

Collaborative sensemaking between subject matter experts and tool experts, facilitated by visual analytics, significantly improves the understanding of complex aviation safety data.

Summit (Simon Fraser University) · 2011

Key Findings

01Paired analysis provides rich data on human analytical reasoning processes.
02This collaborative method supports the development of better analytical tools.
03The approach can be used to enhance analyst training.

Application

Design takeaway

How to apply

When designing data analysis tools, especially for safety-critical domains, consider creating interfaces that enable and record collaborative sessions between users with different types of expertise.

Project actions

01Consider how different users might collaborate on a design task.
02Think about how visual tools can support group analysis.
03Explore methods for capturing user interaction and reasoning during design.

Method & Evidence

AimHow can a collaborative sensemaking approach involving subject matter experts and tool experts, supported by visual analytics, improve the analysis of aviation safety data?

MethodQualitative research, Case study, Method development and evaluation

ProcedureDeveloped and evaluated a method called 'paired analysis' where subject matter experts and tool experts worked together to analyze aircraft maintenance and safety data. This involved interviews, pilot studies, and analytic sessions conducted during an industry internship.

ContextAviation safety, Aircraft maintenance data analysis

Variables

IVCollaborative approach (paired analysis vs. individual analysis)

DVQuality of insights, understanding of data, effectiveness of analytical tools

CVType of data analyzed, visual analytics interface used, expertise of participants

Strengths & Limitations

Strengths

+Focuses on the cognitive and collaborative aspects of analysis.
+Developed and evaluated a novel research method.
+Grounded in real-world industry application.

Limitations

It can be challenging to recruit and coordinate multiple expert participants for collaborative sessions. Capturing and analyzing the rich data from these sessions requires significant effort.

Reliability & validity

The reliability of the paired analysis method would depend on the consistency of the data collected across different dyads and data sets. Validity is supported by the use of real-world data and expert participants, but the method's specific application to aviation safety might limit its external validity.

Think critically

To what extent can the benefits of paired analysis be generalized to design projects where the 'tool expert' is also the primary designer, and the 'subject matter expert' is the end-user?

Design Principles

"Facilitate collaborative sensemaking by integrating diverse expert perspectives within the design and analysis of interactive visual interfaces."

This approach acknowledges that effective data analysis and tool design require a deep understanding of both the domain knowledge and the capabilities of the analytical tools. By bringing these perspectives together, design teams can create more intuitive and powerful visual analytics systems that directly address user needs and improve safety outcomes.

What This Means for Your Design

When people who know a lot about a subject (like pilots) work with people who know a lot about computers and data tools, they can understand safety information much better. This helps make better tools and training.

How to use in your project

1.Use the concept of paired analysis to justify involving different stakeholders in your design process.
2.Explain how collaborative sensemaking can lead to more robust design solutions.

Add to My Project

Quick Cite

Paragraph starter

The research on paired analysis highlights the value of collaborative sensemaking between subject matter experts and tool experts for enhancing understanding in complex domains like aviation safety. This approach, which combines domain knowledge with technical expertise through interactive visual interfaces, yields rich data on analytical reasoning. This insight informs the design of more effective analytical tools and training programs by emphasizing the importance of integrating diverse perspectives to achieve deeper insights and improve outcomes.

Source

Summit (Simon Fraser University)

Visual analytics for aviation safety: A collaborative approach to sensemaking

journal · 2011

View source

Questions About This Research

What does the research say about paired analysis: enhancing aviation safety sensemaking through expert collaboration?: Design interactive visual analytics systems that actively support and capture the collaborative reasoning process between domain experts and tool specialists to uncover deeper insights and improve system design. Evidence: Summit (Simon Fraser University) (2011).
Why does "Paired Analysis: Enhancing Aviation Safety Sensemaking Through Expert Collaboration" matter for design?: This approach acknowledges that effective data analysis and tool design require a deep understanding of both the domain knowledge and the capabilities of the analytical tools. By bringing these perspectives together, design teams can create more intuitive and powerful visual analytics systems that directly address user needs and improve safety outcomes.
How can designers apply this research?: Design interactive visual analytics systems that actively support and capture the collaborative reasoning process between domain experts and tool specialists to uncover deeper insights and improve system design.
What were the main findings?: Paired analysis provides rich data on human analytical reasoning processes.. This collaborative method supports the development of better analytical tools.. The approach can be used to enhance analyst training.
What research method was used?: Qualitative research, Case study, Method development and evaluation.
How strong is the evidence?: Evidence strength is rated Strong effect, based on a 2011 journal from Summit (Simon Fraser University).
What should I do differently in my next project?: When designing data analysis tools, especially for safety-critical domains, consider creating interfaces that enable and record collaborative sessions between users with different types of expertise.
What are the limitations?: The study was conducted within a specific industry context (aviation) and may not be directly generalizable to all domains. The effectiveness of the paired analysis method might depend on the specific expertise and dynamics of the dyad.