Short answer

Designers should actively seek and integrate empirical evidence into their programming language design process, moving beyond intuition and convention to validate design choices.

Field: Innovation & Design
Source: Jyväskylä University Digital Archive (University of Jyväskylä) (2015)
Method: Philosophical analysis and systematic mapping study
Evidence: Moderate effect

Programming language design can be significantly improved by adopting an evidence-based paradigm, drawing on empirical studies to resolve design uncertainties. This innovation & design research insight is drawn from a 2015 study published in Jyväskylä University Digital Archive (University of Jyväskylä). Using Philosophical analysis and systematic mapping study, researchers explored how this design variable affects real-world outcomes. The key design takeaway: Designers should actively seek and integrate empirical evidence into their programming language design process, moving beyond intuition and convention to validate design choices.

Study

Innovation & DesignHigh ImpactModerate effect

Evidence-Based Design Principles for Programming Languages

Programming language design can be significantly improved by adopting an evidence-based paradigm, drawing on empirical studies to resolve design uncertainties.

Jyväskylä University Digital Archive (University of Jyväskylä) · 2015

Key Findings

01Evidence in design is defined as observations that strengthen an argument.
02There is a limited but existing body of empirical evidence applicable to programming language design.
03An evidence-based programming language design (EB-PLD) process can be structured in five steps to address design uncertainties.
04Cohen's kappa is not the most appropriate metric for evaluating coder agreement in systematic secondary studies; Scott's pi, Fleiss' kappa, or Krippendorff's alpha are better alternatives.

Application

Design takeaway

Designers should actively seek and integrate empirical evidence into their programming language design process, moving beyond intuition and convention to validate design choices.

How to apply

When designing a new programming language or feature, systematically search for and analyze existing empirical studies on language design principles, syntax, semantics, and usability. Use this evidence to inform design choices and validate hypotheses.

Project actions

01When researching design choices for your project, look for empirical studies or user testing results.
02Clearly define what constitutes 'evidence' for your specific design problem.

Method & Evidence

AimWhat are the core components and practical implementation of an evidence-based approach to programming language design, and how can existing research be leveraged to inform this process?

MethodPhilosophical analysis and systematic mapping study

ProcedureThe research involved a philosophical exploration of the concept of evidence in design and a systematic review to identify and map existing empirical evidence relevant to programming language design. It also evaluated the suitability of statistical measures for inter-rater reliability in secondary research.

ContextProgramming language design and software engineering research

Variables

IVAdoption of an evidence-based design process

DVQuality and effectiveness of programming language design

CVComplexity of the programming language, target audience, existing design paradigms

Strengths & Limitations

Strengths

+Provides a philosophical grounding for evidence-based design.
+Offers a structured approach (EB-PLD) for integrating evidence into design practice.

Limitations

It can be challenging to find direct empirical evidence for very specific or novel design problems, and interpreting research findings accurately requires careful consideration.

Reliability & validity

The validity of the findings relies on the thoroughness of the systematic mapping study and the philosophical rigor of the explication. Reliability in the context of the study's evaluation of statistical methods is addressed by recommending alternative metrics.

Think critically

How can designers ensure that the evidence they use is truly representative and not biased towards specific user groups or contexts?

Design Principles

"Design decisions should be supported by empirical evidence and systematic evaluation."

This approach moves beyond subjective intuition and established conventions, enabling designers to make more informed decisions supported by observable data. By systematically evaluating design choices, practitioners can create more effective, user-friendly, and robust programming languages.

What This Means for Your Design

Instead of just guessing what makes a good programming language, designers should look at actual studies and data to make informed decisions.

How to use in your project

1.Use the concept of evidence-based design to justify your design choices, referencing empirical data or user research findings.
2.Discuss how you sought and applied evidence to resolve design challenges in your project.

Add to My Project

Quick Cite

Paragraph starter

This design project adopts an evidence-based approach, drawing on empirical findings to inform critical design decisions. By systematically reviewing existing research related to [specific design area, e.g., user interface conventions, cognitive load in syntax], we aimed to move beyond subjective preferences and establish a data-driven foundation for our design choices, ensuring greater effectiveness and usability.

Source

Jyväskylä University Digital Archive (University of Jyväskylä)

Evidence-based programming language design : a philosophical and methodological exploration

journal · 2015

View source

Questions About This Research

What does the research say about evidence-based design principles for programming languages?: Designers should actively seek and integrate empirical evidence into their programming language design process, moving beyond intuition and convention to validate design choices. Evidence: Jyväskylä University Digital Archive (University of Jyväskylä) (2015).
Why does "Evidence-Based Design Principles for Programming Languages" matter for design?: This approach moves beyond subjective intuition and established conventions, enabling designers to make more informed decisions supported by observable data. By systematically evaluating design choices, practitioners can create more effective, user-friendly, and robust programming languages.
How can designers apply this research?: Designers should actively seek and integrate empirical evidence into their programming language design process, moving beyond intuition and convention to validate design choices.
What were the main findings?: Evidence in design is defined as observations that strengthen an argument.. There is a limited but existing body of empirical evidence applicable to programming language design.. An evidence-based programming language design (EB-PLD) process can be structured in five steps to address design uncertainties.. Cohen's kappa is not the most appropriate metric for evaluating coder agreement in systematic secondary studies; Scott's pi, Fleiss' kappa, or Krippendorff's alpha are better alternatives.
What research method was used?: Philosophical analysis and systematic mapping study.
How strong is the evidence?: Evidence strength is rated Moderate effect, based on a 2015 journal from Jyväskylä University Digital Archive (University of Jyväskylä).
What should I do differently in my next project?: When designing a new programming language or feature, systematically search for and analyze existing empirical studies on language design principles, syntax, semantics, and usability. Use this evidence to inform design choices and validate hypotheses.
What are the limitations?: The study acknowledges that the practical usefulness of the proposed EB-PLD process was not within its scope to demonstrate, and the existing evidence base is currently limited.