Short answer

Integrate conversational AI tools into your research workflow to accelerate data discovery, but always critically evaluate the AI's output for accuracy and relevance.

Field
Innovation & Design
Source
arXiv (Cornell University) (2023)
Method
Qualitative research through workshops.
Evidence
Moderate effect

Large language models, when used as conversational agents, can significantly aid users in identifying and understanding relevant datasets for their design projects. This innovation & design research insight is drawn from a 2023 study published in arXiv (Cornell University). Using Qualitative research through workshops., researchers explored how this design variable affects real-world outcomes. The key design takeaway: Integrate conversational AI tools into your research workflow to accelerate data discovery, but always critically evaluate the AI's output for accuracy and relevance.

Study
Innovation & DesignRecentModerate effect

Conversational AI enhances data discovery by providing contextual recommendations and analysis.

Large language models, when used as conversational agents, can significantly aid users in identifying and understanding relevant datasets for their design projects.

arXiv (Cornell University) · 2023

01

Key Findings

  • 01CGAIs can suggest relevant datasets to users.
  • 02CGAIs can provide reasoning for dataset recommendations.
  • 03CGAIs can support sensemaking activities related to datasets.
  • 04CGAIs can assist in dataset analysis and manipulation.
  • 05CGAIs may suggest fictional datasets or perform inaccurate analysis.
02

Application

Design takeaway

Integrate conversational AI tools into your research workflow to accelerate data discovery, but always critically evaluate the AI's output for accuracy and relevance.

How to apply

When starting a new design project that requires data, experiment with conversational AI tools to generate initial dataset suggestions and summaries. Formulate specific prompts to guide the AI towards your project's needs.

Project actions

  • 01Use AI to brainstorm potential data sources for your design project.
  • 02Ask the AI to explain why a particular dataset is relevant to your problem.
  • 03Be aware that AI can sometimes make up information, so always verify data sources and findings.
03

Method & Evidence

AimTo investigate the potential of conversational generative AI (CGAI) to assist users in the process of data discovery and sensemaking.
MethodQualitative research through workshops.
ProcedureConducted three workshops with data users to explore their experiences and identify emerging practices when using CGAIs for data discovery.
ContextData discovery and data science workflows.

Variables

IVUse of conversational generative AI for data discovery.
DVRelevance and usefulness of suggested datasets, efficiency of data discovery process, quality of sensemaking activities.
CVUser's prior knowledge of data discovery, specific domain of the design project.
04

Strengths & Limitations

Strengths

  • +Explores a novel application of emerging AI technology in a practical design context.
  • +Identifies emerging practices and provides a model for future research.

Limitations

The AI might not always understand the nuances of your specific design problem, and its suggestions might be too general or even incorrect.

Reliability & validity

The findings are based on qualitative data from workshops, which may be subject to researcher bias and limited generalizability. The validity of AI-generated information is a key concern.

Think critically

How can designers ensure the ethical use of AI-generated data insights and mitigate the risks associated with AI hallucinations in their design process?

05

Design Principles

"Leverage AI-assisted discovery with critical human oversight."

This capability streamlines the initial stages of research and development by reducing the time spent searching for and evaluating data. By offering reasoned suggestions and documentation-like information, these AI tools empower designers and researchers to make more informed decisions about data utilization early in the design process.

06

What This Means for Your Design

AI chatbots can help you find and understand data for your projects by suggesting datasets and explaining why they might be useful, but you still need to check if the information is correct.

How to use in your project

  • 1.Document your use of conversational AI for data discovery, including the prompts you used and how the AI's suggestions informed your research.
  • 2.Discuss the benefits and limitations of using AI in your data gathering process.
07

Add to My Project

08

Quick Cite

Paragraph starter

Conversational AI tools were explored as a method for accelerating data discovery. By prompting the AI with project requirements, relevant datasets were suggested along with explanations for their utility. While this approach proved efficient in identifying potential data sources, critical evaluation of the AI's output was maintained to mitigate risks of inaccurate or fabricated information, ensuring the integrity of the research foundation.

09

Source

arXiv (Cornell University)

Prompting Datasets: Data Discovery with Conversational Agents

journal · 2023

View source

Questions About This Research

What does the research say about conversational ai enhances data discovery by providing contextual recommendations and analysis?
Integrate conversational AI tools into your research workflow to accelerate data discovery, but always critically evaluate the AI's output for accuracy and relevance. Evidence: arXiv (Cornell University) (2023).
Why does "Conversational AI enhances data discovery by providing contextual recommendations and analysis." matter for design?
This capability streamlines the initial stages of research and development by reducing the time spent searching for and evaluating data. By offering reasoned suggestions and documentation-like information, these AI tools empower designers and researchers to make more informed decisions about data utilization early in the design process.
How can designers apply this research?
Integrate conversational AI tools into your research workflow to accelerate data discovery, but always critically evaluate the AI's output for accuracy and relevance.
What were the main findings?
CGAIs can suggest relevant datasets to users.. CGAIs can provide reasoning for dataset recommendations.. CGAIs can support sensemaking activities related to datasets.. CGAIs can assist in dataset analysis and manipulation.
What research method was used?
Qualitative research through workshops..
How strong is the evidence?
Evidence strength is rated Moderate effect, based on a 2023 journal from arXiv (Cornell University).
What should I do differently in my next project?
When starting a new design project that requires data, experiment with conversational AI tools to generate initial dataset suggestions and summaries. Formulate specific prompts to guide the AI towards your project's needs.
What are the limitations?
The study acknowledges limitations in current web capabilities of CGAIs and the potential for AI to generate inaccurate or fictional data.