Multi-Stakeholder App Evaluation Yields Distinct Success Metrics
Different stakeholder groups (end-users, owners, developers) prioritize distinct criteria when evaluating the success of a mobile application.
Digital Commons - University of South Florida (University of South Florida) · 2017
Key Findings
- 01End-users, owners (lawyers), and software developers have different priorities and evaluation criteria for mobile app success.
- 02A comprehensive evaluation strategy must incorporate methods tailored to each stakeholder group to capture a holistic view of application success.
Application
Design takeaway
When designing and evaluating any product, especially software, ensure that the success metrics and evaluation methods are tailored to the distinct needs and priorities of all key user and stakeholder groups.
How to apply
Before finalizing a design, conduct separate feedback sessions or surveys with end-users, business owners/clients, and the development team to understand their unique success criteria.
Project actions
- 01Clearly define all potential user and stakeholder groups for your design project.
- 02Develop specific questions for each group that address their unique needs and expectations.
Method & Evidence
Variables
Strengths & Limitations
Strengths
- +Addresses the complexity of evaluating products with multiple user bases.
- +Employs a range of evaluation methods for a richer dataset.
Limitations
It can be challenging to recruit and gather meaningful data from multiple distinct stakeholder groups within the constraints of a typical design project.
Reliability & validity
The use of multiple evaluation methods (focus groups, surveys, real-life tests) enhances the validity of the findings by triangulating data. Reliability would depend on the consistency of responses within each method and across similar participants.
Think critically
How might the differing priorities of stakeholders lead to design compromises, and how can these be managed effectively?
Design Principles
"Holistic stakeholder evaluation is essential for product success."
Understanding and addressing the varied needs and expectations of all key stakeholders is crucial for the successful adoption and impact of a designed product. Ignoring any group can lead to underutilization or outright failure, regardless of the product's technical merit.
What This Means for Your Design
Different people who use or benefit from an app (like the people using it, the people who own it, and the people who made it) care about different things when deciding if it's good.
How to use in your project
- 1.Use this research to justify the need for diverse user testing and stakeholder consultation in your design process.
Add to My Project
Quick Cite
(2017). CrashApp™ –Concurrent Multiple Stakeholder Evaluation of a DSR Artefact. Digital Commons - University of South Florida (University of South Florida). Retrieved from https://designdex.org/study/924fe2e4-9c52-4e39-9e15-bb1909917042/multi-stakeholder-app-evaluation-yields-distinct-success-metrics
Paragraph starter
The evaluation of the CrashApp™ demonstrated that different stakeholder groups, including end-users, owners, and developers, possess distinct criteria for assessing mobile application success. This highlights the critical need for design projects to adopt a multi-faceted evaluation approach, employing methods tailored to the unique perspectives and priorities of each identified stakeholder group to ensure comprehensive product validation and optimize adoption.
Source
Digital Commons - University of South Florida (University of South Florida)
CrashApp™ –Concurrent Multiple Stakeholder Evaluation of a DSR Artefact
journal · 2017
View sourceQuestions about this research
- What does the research say about multi-stakeholder app evaluation yields distinct success metrics?
- When designing and evaluating any product, especially software, ensure that the success metrics and evaluation methods are tailored to the distinct needs and priorities of all key user and stakeholder groups. Evidence: Digital Commons - University of South Florida (University of South Florida) (2017).
- Why does "Multi-Stakeholder App Evaluation Yields Distinct Success Metrics" matter for design?
- Understanding and addressing the varied needs and expectations of all key stakeholders is crucial for the successful adoption and impact of a designed product. Ignoring any group can lead to underutilization or outright failure, regardless of the product's technical merit.
- How can designers apply this research?
- When designing and evaluating any product, especially software, ensure that the success metrics and evaluation methods are tailored to the distinct needs and priorities of all key user and stakeholder groups.
- What were the main findings?
- End-users, owners (lawyers), and software developers have different priorities and evaluation criteria for mobile app success.. A comprehensive evaluation strategy must incorporate methods tailored to each stakeholder group to capture a holistic view of application success.
- What research method was used?
- Naturalistic evaluation.
- How strong is the evidence?
- Evidence strength is rated Moderate effect, based on a 2017 journal from Digital Commons - University of South Florida (University of South Florida).
- What should I do differently in my next project?
- Before finalizing a design, conduct separate feedback sessions or surveys with end-users, business owners/clients, and the development team to understand their unique success criteria.
- What are the limitations?
- The specific context of a legal assistance app may limit generalizability to other domains.
- Is there evidence that success metrics affects design outcomes?
- The study found that clients, lawyers, and developers each have unique perspectives on what makes a mobile app successful, highlighting the need for diverse evaluation approaches. Understanding and addressing the varied needs and expectations of all key stakeholders is crucial for the successful adoption and impact of Source: Digital Commons - University of South Florida (University of South Florida) (2017).
- Where does this all key research apply?
- Mobile application development and evaluation It sits within user-centred design research on designdex.org.
Related research topics
success metrics design research · evidence on success metrics · does success metrics improve design outcomes · all key studies for designers · success metrics and all key findings · user-centred design research evidence