Short answer

Designers should consider incorporating observational methods and tools that capture implicit behaviours when evaluating user performance or designing training and assessment systems.

Field: User-Centred Design
Source: ePrints Soton (University of Southampton) (2014)
Method: Hybrid qualitative research method combining video analysis with 'Think Aloud' protocols.
Evidence: Strong effect

Observational video analysis, enhanced by 'Think Aloud' commentaries and a specialized tool, can uncover subtle, tacit indicators of competence that are difficult to articulate through traditional assessment methods. This user-centred design research insight is drawn from a 2014 study published in ePrints Soton (University of Southampton). Using Hybrid qualitative research method combining video analysis with 'think aloud' protocols., researchers explored how this design variable affects real-world outcomes. The key design takeaway: Designers should consider incorporating observational methods and tools that capture implicit behaviours when evaluating user performance or designing training and assessment systems.

Study

User-Centred DesignHigh ImpactStrong effect

Tacit Indicators of Competence Revealed Through Video Analysis

ePrints Soton (University of Southampton) · 2014

Key Findings

01Competence is an example of tacitly understood behaviour.
02Traditional research methods are limited in capturing tacit competence.
03Video-View-Point successfully revealed four tacit indicators of competence: communication, processing clinical information, being in the right place, and being proactive.
04The developed method offers potential for behavioural analysis in other domains.

Application

Design takeaway

Designers should consider incorporating observational methods and tools that capture implicit behaviours when evaluating user performance or designing training and assessment systems.

How to apply

When designing training modules or performance evaluation systems, consider how to observe and analyze subtle, non-verbal cues that indicate expertise, rather than relying solely on explicit self-reporting or objective metrics.

Project actions

01When observing users, look for subtle actions or decisions that seem to indicate expertise, even if the user doesn't explicitly mention them.
02Consider using video recording and think-aloud protocols to capture user behaviour and thought processes in your own design projects.

Method & Evidence

AimTo develop and validate a method for revealing tacit indicators of competence in student nurses through video analysis.

MethodHybrid qualitative research method combining video analysis with 'Think Aloud' protocols.

ProcedureStudent nurses were observed in simulated practice scenarios. They provided 'Think Aloud' commentaries while their actions were recorded on video. A bespoke software tool, Video-View-Point, was developed to facilitate the analysis of these multimedia data, allowing researchers to 'point' at specific subjects of interest within the video and link them to commentary. This was compared against eye-tracking technology.

ContextHealthcare education, specifically nursing simulation.

Variables

IVObservational video analysis with 'Think Aloud' commentary (and the Video-View-Point tool).

DVIdentification of tacit indicators of competence.

CVSimulated practice scenarios, student nurse participants.

Strengths & Limitations

Strengths

+Development of an innovative hybrid research method.
+Successful identification of tacit indicators of competence.

Limitations

It can be challenging to get users to 'think aloud' naturally, and analyzing video data can be time-consuming.

Reliability & validity

The study's validity is supported by the comparison with eye-tracking technology and the successful identification of specific indicators. Reliability would depend on the consistency of the analysis process across different researchers and datasets.

Think critically

How might the 'tacit indicators' identified in this study be generalized to other domains beyond nursing, and what challenges might arise in their identification?

Design Principles

"Tacit knowledge, when made visible through appropriate observation and analysis, can significantly inform the design of effective learning and assessment systems."

Understanding tacit knowledge is crucial for designing effective training and assessment programs. By revealing these often-unspoken cues, designers can create more targeted feedback, develop more realistic simulations, and ultimately improve the development of expertise in complex fields.

What This Means for Your Design

Sometimes people know how to do things well without being able to explain exactly how they do it. This research found a way to watch videos of people doing tasks and listen to their thoughts at the same time to figure out these 'secret' skills.

How to use in your project

1.Use this research to justify the inclusion of observational methods in your design project, especially when assessing user performance or skill acquisition.

Add to My Project

Quick Cite

Paragraph starter

This research highlights the importance of identifying tacit indicators of competence, which are often not explicitly articulated by users. By employing observational methods such as video analysis combined with 'think aloud' protocols, design projects can uncover these subtle yet critical behaviours, leading to more effective design of training and assessment tools.

Source

ePrints Soton (University of Southampton)

'Video-View-Point' - Video analysis to reveal tacit indicators of student nurse competence

journal · 2014

View source

Questions About This Research

What does the research say about tacit indicators of competence revealed through video analysis?: Designers should consider incorporating observational methods and tools that capture implicit behaviours when evaluating user performance or designing training and assessment systems. Evidence: ePrints Soton (University of Southampton) (2014).
Why does "Tacit Indicators of Competence Revealed Through Video Analysis" matter for design?: Understanding tacit knowledge is crucial for designing effective training and assessment programs. By revealing these often-unspoken cues, designers can create more targeted feedback, develop more realistic simulations, and ultimately improve the development of expertise in complex fields.
How can designers apply this research?: Designers should consider incorporating observational methods and tools that capture implicit behaviours when evaluating user performance or designing training and assessment systems.
What were the main findings?: Competence is an example of tacitly understood behaviour.. Traditional research methods are limited in capturing tacit competence.. Video-View-Point successfully revealed four tacit indicators of competence: communication, processing clinical information, being in the right place, and being proactive.. The developed method offers potential for behavioural analysis in other domains.
What research method was used?: Hybrid qualitative research method combining video analysis with 'Think Aloud' protocols..
How strong is the evidence?: Evidence strength is rated Strong effect, based on a 2014 journal from ePrints Soton (University of Southampton).
What should I do differently in my next project?: When designing training modules or performance evaluation systems, consider how to observe and analyze subtle, non-verbal cues that indicate expertise, rather than relying solely on explicit self-reporting or objective metrics.
What are the limitations?: The study was exploratory, and the sample size was not specified. The effectiveness of the bespoke software was tested against eye-tracking, but further validation across different contexts and user groups would be beneficial.