Short answer

Incorporate learning analytics data alongside user surveys to gain deeper insights into how users interact with and perceive your designs, enabling more targeted improvements.

Field: User-Centred Design
Source: Baltic Journal of Modern Computing (2022)
Method: Comparative experimental study
Sample: Limited number of courses and a small sample (specific numbers not provided in the abstract).
Evidence: Moderate effect

Integrating learning analytics data with survey feedback provides a more robust and nuanced evaluation of e-learning course design quality compared to relying on either method alone. This user-centred design research insight is drawn from a 2022 study published in Baltic Journal of Modern Computing. Using Comparative experimental study with Limited number of courses and a small sample (specific numbers not provided in the abstract)., researchers explored how this design variable affects real-world outcomes. The key design takeaway: Incorporate learning analytics data alongside user surveys to gain deeper insights into how users interact with and perceive your designs, enabling more targeted improvements.

Study

User-Centred DesignHigh ImpactModerate effect

Learning Analytics Data Significantly Enhances Course Design Quality Assessment

Integrating learning analytics data with survey feedback provides a more robust and nuanced evaluation of e-learning course design quality compared to relying on either method alone.

Baltic Journal of Modern Computing · 2022

Key Findings

01Learning analytics offers significant benefits for course design improvement.
02A framework combining learning analytics and survey data provides a more comprehensive interpretation of course design quality than learning analytics alone.
03Challenges exist in the implementation and interpretation of learning analytics for course design.

Application

Design takeaway

Incorporate learning analytics data alongside user surveys to gain deeper insights into how users interact with and perceive your designs, enabling more targeted improvements.

How to apply

When evaluating an e-learning platform or educational resource, collect data on user navigation, time spent on modules, completion rates, and error occurrences, and correlate this with direct user feedback from surveys to identify design strengths and weaknesses.

Project actions

01When designing an online learning experience, think about what data you can collect to see how users interact with it.
02Plan how you will combine this data with direct feedback from users to make your design decisions.

Method & Evidence

AimTo investigate the effectiveness of using learning analytics data, in conjunction with survey feedback, for improving the quality assessment and design of e-learning courses.

MethodComparative experimental study

ProcedureA pilot experiment was conducted with two cases. Case 1 evaluated course design quality using only learning analytics data. Case 2 evaluated course design quality using a framework that combined learning analytics data with survey responses based on specific criteria. The results from both cases were compared.

SampleLimited number of courses and a small sample (specific numbers not provided in the abstract).

ContextE-learning course design and quality improvement in higher education.

Variables

IVMethod of course design quality assessment (learning analytics only vs. learning analytics + survey framework).

DVCourse design quality.

CVNumber of courses and sample size (though noted as a limitation).

Strengths & Limitations

Strengths

+Presents a novel framework for course design evaluation.
+Highlights the practical benefits and challenges of learning analytics in design.

Limitations

The study's findings might be specific to the context of higher education e-learning and may not directly apply to all design projects.

Reliability & validity

The study's validity is enhanced by comparing two distinct evaluation methods. Reliability could be improved with a larger sample size and more courses.

Think critically

How might the specific criteria used in the survey framework in Case 2 have influenced the perceived quality of the course design, and could different criteria yield different results?

Design Principles

"Data-informed design decisions, leveraging both behavioral analytics and user feedback, lead to more effective and user-centered outcomes."

This approach allows designers to move beyond subjective opinions and gain objective insights into user interaction and engagement patterns. By understanding how learners actually navigate and interact with course materials, designers can identify specific areas for improvement, leading to more effective and engaging educational experiences.

What This Means for Your Design

This study shows that looking at how people use an online course (learning analytics) and asking them what they think (surveys) together gives a better picture of how to make the course better than just doing one or the other.

How to use in your project

1.Use this research to justify the use of mixed-methods evaluation in your design project, combining analytics with user surveys.
2.Cite this study when discussing how to gather comprehensive data for design improvement.

Add to My Project

Quick Cite

Paragraph starter

The integration of learning analytics with user feedback offers a powerful methodology for enhancing design quality. Research by Žilinskienė (2022) highlights that a combined approach, utilizing both behavioral data from learning analytics and subjective input from surveys, provides a more comprehensive understanding of user interaction and design effectiveness in e-learning contexts, leading to more informed design improvements.

Source

Baltic Journal of Modern Computing

Exploring Learning Analytics for the Course Design Improvement: The Results of a Pilot Experiment

journal · 2022

View source

Questions About This Research

What does the research say about learning analytics data significantly enhances course design quality assessment?: Incorporate learning analytics data alongside user surveys to gain deeper insights into how users interact with and perceive your designs, enabling more targeted improvements. Evidence: Baltic Journal of Modern Computing (2022).
Why does "Learning Analytics Data Significantly Enhances Course Design Quality Assessment" matter for design?: This approach allows designers to move beyond subjective opinions and gain objective insights into user interaction and engagement patterns. By understanding how learners actually navigate and interact with course materials, designers can identify specific areas for improvement, leading to more effective and engaging educational experiences.
How can designers apply this research?: Incorporate learning analytics data alongside user surveys to gain deeper insights into how users interact with and perceive your designs, enabling more targeted improvements.
What were the main findings?: Learning analytics offers significant benefits for course design improvement.. A framework combining learning analytics and survey data provides a more comprehensive interpretation of course design quality than learning analytics alone.. Challenges exist in the implementation and interpretation of learning analytics for course design.
What research method was used?: Comparative experimental study with Limited number of courses and a small sample (specific numbers not provided in the abstract)..
How strong is the evidence?: Evidence strength is rated Moderate effect, based on a 2022 journal from Baltic Journal of Modern Computing.
What should I do differently in my next project?: When evaluating an e-learning platform or educational resource, collect data on user navigation, time spent on modules, completion rates, and error occurrences, and correlate this with direct user feedback from surveys to identify design strengths and weaknesses.
What are the limitations?: The pilot experiment was limited by the number of courses and the sample size, which may affect the generalizability of the findings.