Short answer

Explore opportunities to incorporate haptic feedback, spatial audio, or even olfactory cues into digital products and services to create more immersive and impactful user journeys.

Field
User-Centred Design
Source
ACM Computing Surveys (2018)
Method
Literature Review and Conceptual Framework Development
Evidence
Moderate effect

Integrating multiple sensory channels in digital media can significantly enhance user engagement and interaction by creating more immersive and contextually relevant experiences. This user-centred design research insight is drawn from a 2018 study published in ACM Computing Surveys. Using Literature review and conceptual framework development, researchers explored how this design variable affects real-world outcomes. The key design takeaway: Explore opportunities to incorporate haptic feedback, spatial audio, or even olfactory cues into digital products and services to create more immersive and impactful user journeys.

Study
User-Centred DesignHigh ImpactModerate effect

Mulsemedia: Engaging Multiple Senses for Richer User Experiences

Integrating multiple sensory channels in digital media can significantly enhance user engagement and interaction by creating more immersive and contextually relevant experiences.

ACM Computing Surveys · 2018

01

Key Findings

  • 01Mulsemedia systems offer the potential for layered sensory stimulation and interaction.
  • 02The evolution of technology, especially wearables, provides new avenues for mulsemedia implementation.
  • 03Significant challenges remain in effectively integrating multiple senses into cohesive user experiences.
02

Application

Design takeaway

Explore opportunities to incorporate haptic feedback, spatial audio, or even olfactory cues into digital products and services to create more immersive and impactful user journeys.

How to apply

When designing interactive systems, consider how touch, sound, and potentially other senses can be used to convey information, provide feedback, or evoke emotions, moving beyond traditional screen-based interactions.

Project actions

  • 01Consider how different senses can work together to tell a story or provide information.
  • 02Think about the user experience of someone using a device that engages multiple senses simultaneously.
03

Method & Evidence

AimHow can the integration of multiple sensory modalities in digital media (mulsemedia) be leveraged to create more engaging and effective user experiences?
MethodLiterature Review and Conceptual Framework Development
ProcedureThe review analyzed existing research on systems that combine multiple media types with multiple sensory outputs, focusing on the challenges and opportunities in this 'multi-multi' perspective. It identified underexplored areas and proposed new directions for research and development in mulsemedia.
ContextHuman-Computer Interaction, Multimedia Systems, Wearable Technology

Variables

IV["Type and combination of sensory modalities used (e.g., visual only, visual + audio, visual + audio + haptic).","Design of sensory integration (e.g., synchronized, layered, contrasting)."]
DV["User engagement levels (e.g., time spent, interaction frequency).","Perceived immersiveness.","Task performance or efficiency.","Emotional response."]
CV["Content being presented (e.g., video, game, educational material).","User demographics and prior experience.","Environmental conditions (e.g., lighting, noise)."]
04

Strengths & Limitations

Strengths

  • +Provides a comprehensive overview of the mulsemedia field.
  • +Identifies key challenges and future research directions.

Limitations

Implementing and testing multi-sensory systems can be complex and expensive, requiring specialized hardware and software.

Reliability & validity

The reliability of findings from a literature review depends on the quality and breadth of the studies reviewed. Validity is enhanced by the systematic approach to identifying and synthesizing existing research. For experimental replication, ensure consistent sensory output calibration and standardized user testing protocols.

Think critically

What are the ethical considerations of designing deeply immersive multi-sensory experiences, particularly concerning potential addiction or manipulation?

05

Design Principles

"Design for multi-sensory engagement to deepen user connection and understanding."

As technology advances, particularly with the proliferation of wearables, designers have an unprecedented opportunity to move beyond purely visual and auditory interfaces. By thoughtfully incorporating other senses like touch, smell, or even taste, designers can create more compelling and memorable interactions that cater to a wider range of user needs and preferences.

06

What This Means for Your Design

Imagine a video game that not only shows you the action and plays sounds but also vibrates your controller when you get hit or even releases a scent when a character enters a forest. This is called 'mulsemedia,' and it makes digital experiences more real by using more of your senses.

How to use in your project

  • 1.Use the concept of mulsemedia to justify the inclusion of multi-sensory elements in your design proposal.
  • 2.Discuss how your design aims to leverage multiple senses for a richer user experience, referencing the potential benefits outlined in mulsemedia research.
07

Add to My Project

08

Quick Cite

Paragraph starter

The concept of 'mulsemedia' (multiple sensorial media) suggests that integrating multiple sensory channels can significantly enhance user engagement and interaction. By moving beyond traditional visual and auditory interfaces and incorporating elements like haptic feedback or spatial audio, designers can create richer, more immersive, and contextually relevant experiences. This approach acknowledges that users perceive and interact with the world through a combination of senses, and digital experiences can become more impactful by reflecting this multi-modal reality.

09

Source

ACM Computing Surveys

Is Multimedia Multisensorial? - A Review of Mulsemedia Systems

journal · 2018

View source

Questions About This Research

What does the research say about mulsemedia: engaging multiple senses for richer user experiences?
Explore opportunities to incorporate haptic feedback, spatial audio, or even olfactory cues into digital products and services to create more immersive and impactful user journeys. Evidence: ACM Computing Surveys (2018).
Why does "Mulsemedia: Engaging Multiple Senses for Richer User Experiences" matter for design?
As technology advances, particularly with the proliferation of wearables, designers have an unprecedented opportunity to move beyond purely visual and auditory interfaces. By thoughtfully incorporating other senses like touch, smell, or even taste, designers can create more compelling and memorable interactions that cater to a wider range of user needs and preferences.
How can designers apply this research?
Explore opportunities to incorporate haptic feedback, spatial audio, or even olfactory cues into digital products and services to create more immersive and impactful user journeys.
What were the main findings?
Mulsemedia systems offer the potential for layered sensory stimulation and interaction.. The evolution of technology, especially wearables, provides new avenues for mulsemedia implementation.. Significant challenges remain in effectively integrating multiple senses into cohesive user experiences.
What research method was used?
Literature Review and Conceptual Framework Development.
How strong is the evidence?
Evidence strength is rated Moderate effect, based on a 2018 journal from ACM Computing Surveys.
What should I do differently in my next project?
When designing interactive systems, consider how touch, sound, and potentially other senses can be used to convey information, provide feedback, or evoke emotions, moving beyond traditional screen-based interactions.
What are the limitations?
The review highlights that many mulsemedia applications are still in nascent stages, with limited real-world deployment and user testing.