Short answer
Designers should prioritize 'skeptical' user interfaces that encourage users to verify AI suggestions rather than blindly accepting automated security recommendations.
- Field
- User-Centred Design
- Source
- Cybersecurity (2025)
- Method
- Systematic Literature Review
- Sample
- 300+ research papers
- Evidence
- Strong effect
While LLMs automate complex security tasks, they introduce new psychological factors where users may over-rely on AI-generated content that could be subtly malicious. This user-centred design research insight is drawn from a 2025 study published in Cybersecurity. Using Systematic literature review with 300+ research papers, researchers explored how this design variable affects real-world outcomes. The key design takeaway: Designers should prioritize 'skeptical' user interfaces that encourage users to verify AI suggestions rather than blindly accepting automated security recommendations.
Integrating Large Language Models into security interfaces improves threat detection speed but increases user vulnerability to social engineering
While LLMs automate complex security tasks, they introduce new psychological factors where users may over-rely on AI-generated content that could be subtly malicious.
Cybersecurity · 2025
Key Findings
- 01LLMs significantly reduce the technical barrier for non-experts to perform complex security audits.
- 02AI-generated phishing content is significantly more convincing to users than human-generated templates.
- 03Prompt engineering allows attackers to bypass safety filters to generate malicious code.
Application
Design takeaway
Designers should prioritize 'skeptical' user interfaces that encourage users to verify AI suggestions rather than blindly accepting automated security recommendations.
How to apply
When designing a dashboard or app that uses AI, include a 'Confidence Score' or 'Source Verification' tag next to every AI-generated insight to maintain user awareness.
Project actions
- 01If designing a login system or security app, consider how you can warn the user about 'Deepfake' or AI-generated scams.
- 02Focus on the 'Psychological Factors' of your user—how do they feel when an AI tells them their system is safe?
Method & Evidence
Variables
Strengths & Limitations
Strengths
- +Comprehensive overview of current technology
- +Identifies specific 'downstream' risks for designers
Limitations
The paper focuses on the software side; for a DT project, you must relate this back to the physical interface or the user's interaction with the device.
Reliability & validity
High reliability due to the large volume of papers reviewed (300+), though the field of AI moves so fast that specific LLM names may become outdated quickly.
Think critically
If a design is 'too easy' to use, does it actually make the user less safe by making them less observant?
Design Principles
"The Principle of Least Trust: Design interfaces that assume AI-generated output may be flawed or compromised, requiring explicit user confirmation for high-risk actions."
In the context of design, this research bridges the gap between User-Centred Design (design topics) and the psychological factors of Human Factors (design topics). It highlights how the 'usability' of a system can be compromised by the very tools meant to simplify it, specifically through the lens of trust and cognitive load in high-stakes digital environments.
What This Means for Your Design
Using AI to help with security makes things faster, but it also makes it easier for hackers to trick people because the AI can write very convincing fake messages.
How to use in your project
- 1.Cite this when justifying why your app interface needs clear warning labels or multi-step verification for sensitive data.
Add to My Project
Quick Cite
Paragraph starter
According to a 2025 systematic review by Zhang et al., the integration of Large Language Models (LLMs) into digital systems creates a 'dual-use' dilemma. While it improves the usability of security tools for the average user, it also increases the risk of successful social engineering attacks. This suggests that a User-Centred Design approach must balance ease-of-use with critical friction points to ensure users do not over-rely on automated AI advice.
Source
Questions About This Research
- What does the research say about integrating large language models into security interfaces improves threat detection speed but increases user vulnerability to social engineering?
- Designers should prioritize 'skeptical' user interfaces that encourage users to verify AI suggestions rather than blindly accepting automated security recommendations. Evidence: Cybersecurity (2025).
- Why does "Integrating Large Language Models into security interfaces improves threat detection speed but increases user vulnerability to social engineering" matter for design?
- In the context of IB DT, this research bridges the gap between User-Centred Design (Topic 7) and the psychological factors of Human Factors (Topic 1). It highlights how the 'usability' of a system can be compromised by the very tools meant to simplify it, specifically through the lens of trust and cognitive load in high-stakes digital environments.
- How can designers apply this research?
- Designers should prioritize 'skeptical' user interfaces that encourage users to verify AI suggestions rather than blindly accepting automated security recommendations.
- What were the main findings?
- LLMs significantly reduce the technical barrier for non-experts to perform complex security audits.. AI-generated phishing content is significantly more convincing to users than human-generated templates.. Prompt engineering allows attackers to bypass safety filters to generate malicious code.
- What research method was used?
- Systematic Literature Review with 300+ research papers.
- How strong is the evidence?
- Evidence strength is rated Strong effect, based on a 2025 journal from Cybersecurity.
- What should I do differently in my next project?
- When designing a dashboard or app that uses AI, include a 'Confidence Score' or 'Source Verification' tag next to every AI-generated insight to maintain user awareness.
- What are the limitations?
- The study is a review of existing literature and does not conduct new primary user testing on specific interface layouts.