Design Principles for Human-Agent Interaction

Checked 30 Sept 2026. Open license (CC-BY, CC-BY-SA, CC0, public domain): full text indexed and used in synthesis.

AI agents are rapidly evolving into autonomous systems capable of sustained interaction, tool use, and long-term collaboration. Yet their real-world adoption remains limited, suggesting that the key barrier lies not only in technical capability but also in a lack of design knowledge for successful human-agent interaction. This position paper argues that AI agents should not be solely evaluated or deployed based on autonomous task capability alone; because agents interact with, adapt to, influence, and sometimes fail humans, human-agent interaction must be treated as a core design and evaluation target for agentic AI. We present 14 design principles that articulate the ideal human-agent relationship across four interaction stages: initially, during interaction, over time, and when things go wrong. We use these principles to evaluate nine agent systems to illustrate that these design principles can provide actionable guidance for AI design teams to systematically design and evaluate agents that are usable, trustworthy, and effective in real-world interactive settings.

• The emotional reliance on GenAI that Boyd & Harding 2025 document in doctoral supervision—"the emotional support ascribed by students to the technology"—is independently flagged by Zhu et al. 2026 as an unmet design challenge, since "deployment in emotionally sensitive contexts" is among the agent capabilities "not fully captured by existing principles." Trace →

• Supervision and doctoral educationanchor