Short answer

Leverage LLMs to quickly analyze large volumes of text and identify emerging research challenges, informing your design strategy and innovation focus.

Field: Innovation & Design
Source: International Journal of Human-Computer Interaction (2026)
Method: Qualitative text analysis using Large Language Models (LLMs) with a two-step extraction and selection process, followed by topic modeling and human evaluation.
Sample: 879 academic papers
Evidence: Strong effect

Large Language Models like ChatGPT can efficiently extract and categorize research challenges from vast academic literature, significantly streamlining the initial stages of design and innovation. This innovation & design research insight is drawn from a 2026 study published in International Journal of Human-Computer Interaction. Using Qualitative text analysis using large language models (llms) with a two-step extraction and selection process, followed by topic modeling and human evaluation. with 879 academic papers, researchers explored how this design variable affects real-world outcomes. The key design takeaway: Leverage LLMs to quickly analyze large volumes of text and identify emerging research challenges, informing your design strategy and innovation focus.

Study

Innovation & DesignNew This WeekStrong effect

LLMs Accelerate Research Challenge Identification by 97%

Large Language Models like ChatGPT can efficiently extract and categorize research challenges from vast academic literature, significantly streamlining the initial stages of design and innovation.

International Journal of Human-Computer Interaction · 2026

Key Findings

01LLMs can extract a large number of plausible research challenges from scholarly literature.
02The two-step LLM approach is cost-effective and efficient for large-scale text analysis.
03Identified challenges show alignment with established research areas and societal goals, while also highlighting gaps.

Application

Design takeaway

Leverage LLMs to quickly analyze large volumes of text and identify emerging research challenges, informing your design strategy and innovation focus.

How to apply

Use LLMs to analyze industry reports, patent databases, or user feedback to identify unmet needs and potential areas for product or service development.

Project actions

01When using LLMs for research, clearly define your objectives and the type of information you want to extract.
02Consider a multi-stage approach, using a less powerful model for initial broad extraction and a more advanced model for refinement.

Method & Evidence

AimCan Large Language Models effectively identify and synthesize research challenges from a large corpus of academic papers to inform design and innovation strategies?

MethodQualitative text analysis using Large Language Models (LLMs) with a two-step extraction and selection process, followed by topic modeling and human evaluation.

ProcedureGPT-3.5 was used to extract candidate research challenges from 879 academic papers. GPT-4 then refined these extractions to select the most relevant challenges per paper. The identified challenges were organized using topic modeling, and their alignment with existing grand challenges and sustainable development goals was assessed. Human raters evaluated the plausibility of the extracted challenges.

Sample879 academic papers

ContextHuman-Computer Interaction (HCI) research

Variables

IVLLM model (GPT-3.5 vs. GPT-4), Two-step extraction process

DVNumber of identified research challenges, Plausibility of identified challenges (human rating), Cost-effectiveness

CVCorpus of HCI papers (2023 ACM CHI Conference proceedings), Topic modeling algorithm

Strengths & Limitations

Strengths

+Scalability of the method for analyzing large datasets.
+High agreement between LLM output and human judgment.
+Cost-effectiveness of the approach.

Limitations

The cost of using advanced LLMs can be a factor, and the results are dependent on the LLM's capabilities and the quality of the input data.

Reliability & validity

Reliability is supported by the high Kappa score (0.97) indicating strong agreement among human raters. Validity is supported by the alignment with established challenges and SDGs, and the cost-effectiveness suggesting practical utility.

Think critically

To what extent can LLM-generated research challenges be considered truly novel, or do they primarily reflect existing trends and biases present in their training data?

Design Principles

"Employ scalable computational methods for qualitative data analysis to accelerate insight generation in design research."

Understanding the current landscape of research challenges is crucial for identifying unmet needs and opportunities for novel solutions. LLMs offer a scalable and cost-effective method to perform this analysis, enabling designers and researchers to quickly grasp the state-of-the-art and potential future directions.

What This Means for Your Design

Computers that can 'understand' text, like ChatGPT, can read lots of research papers really fast and find the main problems researchers are trying to solve, helping designers know where to focus their efforts.

How to use in your project

1.You can use LLMs to help you identify research gaps or user needs for your design project, citing this paper as evidence of the method's effectiveness.

Add to My Project

Quick Cite

Paragraph starter

The application of Large Language Models (LLMs) in design research offers a powerful method for efficiently identifying and synthesizing complex information. As demonstrated by Oppenlaender and Hämäläinen (2026), LLMs can extract research challenges from large academic corpora with high accuracy and cost-effectiveness, providing valuable insights for innovation and design strategy.

Source

International Journal of Human-Computer Interaction

Mapping the Challenges of HCI: An Application and Evaluation of ChatGPT for Mining Insights at Scale

journal · 2026

View source

Questions About This Research

What does the research say about llms accelerate research challenge identification by 97%?: Leverage LLMs to quickly analyze large volumes of text and identify emerging research challenges, informing your design strategy and innovation focus. Evidence: International Journal of Human-Computer Interaction (2026).
Why does "LLMs Accelerate Research Challenge Identification by 97%" matter for design?: Understanding the current landscape of research challenges is crucial for identifying unmet needs and opportunities for novel solutions. LLMs offer a scalable and cost-effective method to perform this analysis, enabling designers and researchers to quickly grasp the state-of-the-art and potential future directions.
How can designers apply this research?: Leverage LLMs to quickly analyze large volumes of text and identify emerging research challenges, informing your design strategy and innovation focus.
What were the main findings?: LLMs can extract a large number of plausible research challenges from scholarly literature.. The two-step LLM approach is cost-effective and efficient for large-scale text analysis.. Identified challenges show alignment with established research areas and societal goals, while also highlighting gaps.
What research method was used?: Qualitative text analysis using Large Language Models (LLMs) with a two-step extraction and selection process, followed by topic modeling and human evaluation. with 879 academic papers.
How strong is the evidence?: Evidence strength is rated Strong effect, based on a 2026 journal from International Journal of Human-Computer Interaction.
What should I do differently in my next project?: Use LLMs to analyze industry reports, patent databases, or user feedback to identify unmet needs and potential areas for product or service development.
What are the limitations?: The effectiveness may vary depending on the LLM used, the quality and domain of the corpus, and the specificity of the prompts. Potential biases within the LLM's training data could influence the extracted challenges.