Short answer
Build custom evaluation rubrics that weigh cultural harmony and local ethics as equal to technical usability when designing for specific regional populations.
- Field
- User-Centred Design
- Source
- Journal of Physics Conference Series (2020)
- Method
- Instrument development approach with expert judgment (Content Validity Ratio)
- Sample
- 2 experts
- Evidence
- Moderate effect
Integrating indigenous philosophical values and local ethical standards into testing instruments ensures that user interface evaluations measure socio-cultural relevance alongside technical usability. This user-centred design research insight is drawn from a 2020 study published in Journal of Physics Conference Series. Using Instrument development approach with expert judgment (content validity ratio) with 2 experts, researchers explored how this design variable affects real-world outcomes. The key design takeaway: Build custom evaluation rubrics that weigh cultural harmony and local ethics as equal to technical usability when designing for specific regional populations.
Culturally-aligned evaluation frameworks improve the content validity of interface quality assessments
Integrating indigenous philosophical values and local ethical standards into testing instruments ensures that user interface evaluations measure socio-cultural relevance alongside technical usability.
Journal of Physics Conference Series · 2020
Key Findings
- 01Expert consensus validated that a 20-item specialized instrument can effectively measure UI quality within a local cultural framework.
- 02The content validity coefficient of 0.75 indicates a high level of agreement on the instrument's relevance to its intended cultural and technical goals.
- 03Informatics-driven UI metrics can be successfully synthesized with non-technical values like 'Public Ethics' and 'Anti-Corruption' to create a unified evaluation score.
Application
Design takeaway
Build custom evaluation rubrics that weigh cultural harmony and local ethics as equal to technical usability when designing for specific regional populations.
How to apply
When designing for a specific region, identify 3-5 local cultural pillars. Map these to UI elements (e.g., 'Harmony' might map to visual balance and non-intrusive notifications) and have local subject matter experts validate your evaluation checklist before testing with users.
Project actions
- 01Don't just use Nielsen's 10 Heuristics for everything; try defining your own rules based on your user group's specific culture.
- 02Have two different 'experts' (like a teacher and a tech-savvy friend) look at your evaluation questions to see if they make sense.
- 03Quality isn't a single score; it’s a mix of how it looks, how it works, and how it feels to the community.
Method & Evidence
Variables
Strengths & Limitations
Strengths
- +Specifically addresses a gap in existing UX heuristics by incorporating cultural values, enhancing ecological validity for target user groups.
- +Employs an established quantitative method (Content Validity Ratio) for assessing expert judgment, providing a structured approach to validity.
- +Provides a practical, context-specific methodology that can be adapted for other cultural contexts and evaluation needs.
Limitations
Since you likely don't have access to many PhD experts, your 'content validity' will be much lower than a professional study, which you should acknowledge in your evaluation section.
Reliability & validity
The study focuses on content validity, using expert judgment (CVR) as its primary measure. Reliability is not explicitly tested but could be inferred from the consistency of expert ratings if multiple experts were used. The validity is strengthened by aligning the instrument with a specific cultural framework, addressing a known issue with standardized heuristics. However, the limited sample size (2 experts) severely restricts the generalizability of the validity claims and introduces potential bias.
Think critically
If an interface is highly usable but violates a local cultural norm, can it truly be called 'high quality'? How should designers weigh technical efficiency against cultural harmony?
Design Principles
"Socio-Cultural Interface Validity"
Standardized UX heuristics often ignore local cultural values, leading to products that function well but feel culturally discordant. By validating instruments that include values like THK (harmony) and ANEKA (ethics), designers can ensure their interfaces resonate deeply with specific regional user identities and social expectations.
What This Means for Your Design
A good app design isn't just about pretty buttons; it has to align with the values of the people using it. This study shows that we can scientifically prove whether a test for an app's design is actually 'correct' for the culture it serves.
How to use in your project
- 1.Reference this when justifying why you created a custom survey or checklist to test your prototype's cultural suitability.
- 2.Use it to support the 'Evaluation' or 'Analysis' sections of the project when discussing how to measure 'Quality'.
Add to My Project
Quick Cite
Paragraph starter
According to Divayana et al. (2020), integrating local ethical and cultural frameworks into UI evaluation instruments is essential for ensuring the content validity of quality assessments in specific regional contexts.
Source
Journal of Physics Conference Series
Content validity of quality test instrument used for the user interface design of THK-ANEKA-based countenance evaluation application
journal · 2020
View sourceQuestions About This Research
- What does the research say about culturally-aligned evaluation frameworks improve the content validity of interface quality assessments?
- Build custom evaluation rubrics that weigh cultural harmony and local ethics as equal to technical usability when designing for specific regional populations. Evidence: Journal of Physics Conference Series (2020).
- Why does "Culturally-aligned evaluation frameworks improve the content validity of interface quality assessments" matter for design?
- Standardized UX heuristics often ignore local cultural values, leading to products that function well but feel culturally discordant. By validating instruments that include values like THK (harmony) and ANEKA (ethics), designers can ensure their interfaces resonate deeply with specific regional user identities and social expectations.
- How can designers apply this research?
- Build custom evaluation rubrics that weigh cultural harmony and local ethics as equal to technical usability when designing for specific regional populations.
- What were the main findings?
- Expert consensus validated that a 20-item specialized instrument can effectively measure UI quality within a local cultural framework.. The content validity coefficient of 0.75 indicates a high level of agreement on the instrument's relevance to its intended cultural and technical goals.. Informatics-driven UI metrics can be successfully synthesized with non-technical values like 'Public Ethics' and 'Anti-Corruption' to create a unified evaluation score.
- What research method was used?
- Instrument development approach with expert judgment (Content Validity Ratio) with 2 experts.
- How strong is the evidence?
- Evidence strength is rated Moderate effect, based on a 2020 journal from Journal of Physics Conference Series.
- What should I do differently in my next project?
- When designing for a specific region, identify 3-5 local cultural pillars. Map these to UI elements (e.g., 'Harmony' might map to visual balance and non-intrusive notifications) and have local subject matter experts validate your evaluation checklist before testing with users.
- What are the limitations?
- The study relies on a very small expert sample (2), which limits broader statistical generalization, and focuses on a very specific Indonesian cultural context.