Short answer

Don't use predictive models to guarantee how fast a user will complete a task; use them only to rank the theoretical efficiency of different layout options.

Field: User-Centred Design
Source: Academic Publication (2020)
Method: Comparative Benchmarking (Predictive Modeling vs. Oculographic Experiment)
Evidence: Moderate effect

The GOMS model calculates ideal motor execution times, which fail to account for the cognitive visual search load of new users while underestimating the muscle-memory efficiency of experienced ones. This user-centred design research insight is drawn from a 2020 study published in Academic Publication. Using Comparative benchmarking (predictive modeling vs. oculographic experiment), researchers explored how this design variable affects real-world outcomes. The key design takeaway: Don't use predictive models to guarantee how fast a user will complete a task; use them only to rank the theoretical efficiency of different layout options.

Study

User-Centred DesignHigh ImpactModerate effect

Interface familiarity shifts GOMS-TLM accuracy from overestimation to underestimation in mobile interaction speed

The GOMS model calculates ideal motor execution times, which fail to account for the cognitive visual search load of new users while underestimating the muscle-memory efficiency of experienced ones.

Academic Publication · 2020

Key Findings

01First-time users perform significantly slower than GOMS-TLM predictions due to high visual search time.
02Familiar users outperform GOMS-TLM predictions, suggesting the model's touch-operator constants are too conservative for power users.
03GOMS-TLM provides a reliable baseline for relative comparison between designs despite its inability to capture absolute learning speeds.

Application

Design takeaway

Don't use predictive models to guarantee how fast a user will complete a task; use them only to rank the theoretical efficiency of different layout options.

How to apply

When presenting a new UI layout to stakeholders, include a 'Learning Lag' factor for the first 3-5 uses to account for the gap between theoretical GOMS speed and actual gaze-to-touch latency.

Project actions

01Use this for a Comparative study between two navigation styles (e.g., Burger Menu vs. Bottom Bar).
02Identify 'search time' vs 'touch time' in your usability testing.
03Map out a 'Happy Path' and calculate the theoretical number of steps vs actual user steps.

Method & Evidence

AimTo compare the predictive accuracy of the GOMS-TLM (Goals, Operators, Methods, and Selection rules - Touch Level Model) against empirical eye-tracking data in mobile interface interactions.

MethodComparative Benchmarking (Predictive Modeling vs. Oculographic Experiment)

ProcedureParticipants performed specific interaction tasks within a mobile application while their eye movements and completion times were recorded using Tribe Eye Tracking equipment. These results were then compared against a theoretical GOMS-TLM model adjusted for touch-screen interactions (tapping, swiping, etc.).

ContextMobile Application User Interface

Variables

IVInterface familiarity (implied by the shift from overestimation to underestimation)

DVAccuracy of GOMS-TLM predictions compared to empirical eye-tracking data (measures of prediction error, e.g., difference in estimated vs. actual interaction time)

CVSpecific mobile interaction tasks performed, type of mobile interface used, specific eye-tracking equipment used for empirical data collection, definition of GOMS operators and selection rules

Strengths & Limitations

Strengths

+Direct comparison between a theoretical predictive model and empirical user data, providing a robust evaluation of the model's applicability.
+Focuses on a highly relevant context (mobile interaction) and a practical problem (design efficiency benchmarks).
+Highlights a critical nuance in predictive modeling: the impact of user familiarity on accuracy.

Limitations

Students rarely have access to high-end eye-trackers, so they must use screen recordings and 'think-aloud' protocols to estimate when a user is searching vs acting.

Reliability & validity

Reliability: The study's reliability is dependent on the consistency of the eye-tracking equipment's measurements and the precise definition of GOMS 'operators'. Inconsistent measurement of operator times or eye-tracking data would reduce reliability. Validity: The study has strong ecological validity for the specific mobile context studied. However, construct validity might be questioned if the simplified GOMS operators do not accurately represent the cognitive processes involved. External validity is limited by the specific hardware and app used.

Think critically

If a model like GOMS predicts a design is 'perfect,' but a human struggles to use it, is the design at fault or is the model missing a variable?

Design Principles

"Predictive modeling accuracy is proportional to user expertise."

Designers often rely on predictive models to justify layout efficiency, but these models assume 'expert' performance. Understanding when predictive models deviate from real-world eye-tracking data prevents design teams from setting unrealistic performance benchmarks for first-time users.

What This Means for Your Design

A computer-simulated model of how fast someone can use an app is usually wrong for beginners because it ignores the time it takes to look around and find buttons.

How to use in your project

1.Reference this to justify why you conducted user testing instead of just following a 'logical' layout.
2.Use it to explain why your prototype test results were slower than you expected (The Learning Gap).

Add to My Project

Quick Cite

Paragraph starter

According to research by Kompaniets et al. (2020), predictive models like GOMS-TLM often fail to account for the visual searching phase in novice users, leading to an overestimation of interface efficiency.

Source

Academic Publication

GOMS-TLM and Eye Tracking Methods Comparison in the User Interface Interaction Speed Assessing Task

journal · 2020

View source

Questions About This Research

What does the research say about interface familiarity shifts goms-tlm accuracy from overestimation to underestimation in mobile interaction speed?: Don't use predictive models to guarantee how fast a user will complete a task; use them only to rank the theoretical efficiency of different layout options. Evidence: Academic Publication (2020).
Why does "Interface familiarity shifts GOMS-TLM accuracy from overestimation to underestimation in mobile interaction speed" matter for design?: Designers often rely on predictive models to justify layout efficiency, but these models assume 'expert' performance. Understanding when predictive models deviate from real-world eye-tracking data prevents design teams from setting unrealistic performance benchmarks for first-time users.
How can designers apply this research?: Don't use predictive models to guarantee how fast a user will complete a task; use them only to rank the theoretical efficiency of different layout options.
What were the main findings?: First-time users perform significantly slower than GOMS-TLM predictions due to high visual search time.. Familiar users outperform GOMS-TLM predictions, suggesting the model's touch-operator constants are too conservative for power users.. GOMS-TLM provides a reliable baseline for relative comparison between designs despite its inability to capture absolute learning speeds.
What research method was used?: Comparative Benchmarking (Predictive Modeling vs. Oculographic Experiment).
How strong is the evidence?: Evidence strength is rated Moderate effect, based on a 2020 journal from Academic Publication.
What should I do differently in my next project?: When presenting a new UI layout to stakeholders, include a 'Learning Lag' factor for the first 3-5 uses to account for the gap between theoretical GOMS speed and actual gaze-to-touch latency.
What are the limitations?: The study utilized specific eye-tracking hardware which may have different sampling rates than mobile-integrated eye tracking; GOMS results depend heavily on the accuracy of the predefined 'operator' times (e.g., how many milliseconds a 'tap' takes).