the-midnight-paper

“Like Having a Really bad PA”: The Gulf between User Expectation and Experience of Conversational Agents

Authors: Ewa Luger, Abigail Sellen,

ACM CHI 2016, San Jose, CA, USA

Conversational Agents; mental models; evaluation

Whilst we have long been familiar with the notion of computers that speak, the investigative concern within HCI has been upon multimodality rather than dialogue alone, and there is no sense of how such interfaces are used in everyday life.

Multimodality is a theory of communication and social semiotics. It describes communication practices in terms of the textual, aural, linguistic, spatial, and visual resources - or modes - used to compose messages.

The paper present the findings of 14 semi-structured interviews conducted with existing users of CA systems in an effort to understand the current interactional factors affecting everyday use.
On the basis of its analysis, it presents four key areas where current systems fail to support effective user interaction.

It also considers the implications of Norman’s ‘gulfs of execution and evaluation; for the design of future conversational systems.

Most of the research on CA tends towards either technical papers related to architecture, CAs studied in experimental settings, or systems created for specific contexts, such as guiding users around a space, delivering information, or for the support of language learning.

Though each study brings us closer to understanding effective design, without concurrent knowledge of the pragmatics of everyday use, we fail to truly understand dynamics such as how and why such systems are used and “which factors influence acceptance and success in such scenarios”.

The paper seeks to understand user experience of CA systems by answering two simple questions;