Short answer

Implement systems that allow community members to contribute to the quality assurance of machine-translated user-generated content, focusing on optimizing the experience for shorter text segments.

Field: User-Centred Design
Source: Arrow@dit (Dublin Institute of Technology) (2015)
Method: Mixed-methods approach
Evidence: Moderate effect

Community members can effectively post-edit machine-translated user-generated content, particularly for shorter text segments, making it a viable approach for managing large volumes of online information. This user-centred design research insight is drawn from a 2015 study published in Arrow@dit (Dublin Institute of Technology). Using Mixed-methods approach, researchers explored how this design variable affects real-world outcomes. The key design takeaway: Implement systems that allow community members to contribute to the quality assurance of machine-translated user-generated content, focusing on optimizing the experience for shorter text segments.

Study

User-Centred DesignHigh ImpactModerate effect

Lay Post-Editing of Machine-Translated User-Generated Content is Feasible for Short Segments

Arrow@dit (Dublin Institute of Technology) · 2015

Key Findings

01Lay post-editing is a statistically significant feasible concept for machine-translated user-generated content.
02Post-editing is successful for short segments requiring approximately 35% post-editing effort.
03No distinct post-editing patterns were identified for segments requiring more effort.
04Post-editing quality was largely independent of the measured profile characteristics of the post-editors.

Application

Design takeaway

Implement systems that allow community members to contribute to the quality assurance of machine-translated user-generated content, focusing on optimizing the experience for shorter text segments.

How to apply

When designing platforms with user-generated content that needs to be accessible in multiple languages, consider integrating a lay post-editing workflow, perhaps as a tiered quality assurance step.

Project actions

01Consider how user communities can contribute to content quality.
02Explore the trade-offs between machine translation, human editing, and community involvement.

Method & Evidence

AimTo investigate the feasibility and quality of lay post-editing for machine-translated user-generated content within an online community context.

MethodMixed-methods approach

ProcedureThe study involved lay post-editors (community members) correcting machine-translated text from English to German in a technology support forum. Quality was assessed quantitatively through error annotation, domain specialist evaluation, and end-user evaluation. Post-editing behaviour was analyzed qualitatively.

ContextOnline technology support forums

Variables

IVText segment length, post-editing effort required.

DVPost-editing quality, post-editing effort.

CVMachine translation engine, original language, target language, domain of content.

Strengths & Limitations

Strengths

+Addresses a gap in research by focusing on lay post-editors.
+Employs a mixed-methods approach for comprehensive analysis.

Limitations

The quality of edits can be inconsistent, and it can be hard to predict which texts will require more effort to fix.

Reliability & validity

Quantitative measures like error annotation and specialist/end-user evaluations contribute to reliability and validity, though the qualitative aspect might be more subjective. The study notes difficulties in pinpointing reasons for variance, which could impact reliability.

Think critically

How can the variability in lay post-editing quality be mitigated to ensure a consistent user experience across different languages and content types?

Design Principles

"Leverage distributed human intelligence for content quality and localization, particularly in user-generated environments."

This research highlights an opportunity to leverage community expertise for content localization and quality assurance. By understanding the conditions under which lay post-editing is most effective, design teams can develop more efficient workflows for managing multilingual user-generated content, improving accessibility and user experience.

What This Means for Your Design

People who aren't professional translators can help fix machine translations, especially if the text isn't too long. This is a good way to make sure online content is understandable in different languages.

How to use in your project

1.This study can inform the development of user interfaces for collaborative content creation and editing, especially in multilingual contexts.

Add to My Project

Quick Cite

Paragraph starter

This research demonstrates that lay post-editing of machine-translated user-generated content is a feasible approach, particularly for shorter text segments requiring approximately 35% post-editing effort. This suggests that design projects aiming to manage multilingual content can benefit from incorporating community-driven quality assurance mechanisms, optimizing for efficiency in these specific scenarios.

Source

Arrow@dit (Dublin Institute of Technology)

Community post-editing of machine-translated user-generated content

journal · 2015

View source

Questions About This Research

What does the research say about lay post-editing of machine-translated user-generated content is feasible for short segments?: Implement systems that allow community members to contribute to the quality assurance of machine-translated user-generated content, focusing on optimizing the experience for shorter text segments. Evidence: Arrow@dit (Dublin Institute of Technology) (2015).
Why does "Lay Post-Editing of Machine-Translated User-Generated Content is Feasible for Short Segments" matter for design?: This research highlights an opportunity to leverage community expertise for content localization and quality assurance. By understanding the conditions under which lay post-editing is most effective, design teams can develop more efficient workflows for managing multilingual user-generated content, improving accessibility and user experience.
How can designers apply this research?: Implement systems that allow community members to contribute to the quality assurance of machine-translated user-generated content, focusing on optimizing the experience for shorter text segments.
What were the main findings?: Lay post-editing is a statistically significant feasible concept for machine-translated user-generated content.. Post-editing is successful for short segments requiring approximately 35% post-editing effort.. No distinct post-editing patterns were identified for segments requiring more effort.. Post-editing quality was largely independent of the measured profile characteristics of the post-editors.
What research method was used?: Mixed-methods approach.
How strong is the evidence?: Evidence strength is rated Moderate effect, based on a 2015 journal from Arrow@dit (Dublin Institute of Technology).
What should I do differently in my next project?: When designing platforms with user-generated content that needs to be accessible in multiple languages, consider integrating a lay post-editing workflow, perhaps as a tiered quality assurance step.
What are the limitations?: Variability in output quality was noted, and reasons for this variance were difficult to pinpoint. The study did not identify specific post-editing patterns for segments requiring significant effort.