© Private
Findings from Empirical Studies of Real-world Interactions with LLM-based Conversational Systems
Dr. Johanne Trippas (RMIT University)
06. August 2025
11:15 – 12:00
Abstract: The emergence of large language models has transformed the landscape of conversational systems, but our understanding of how users interact with these systems and what they seek to accomplish remains limited. This talk presents findings from two empirical studies investigating real-world interactions with LLM-based and voice-based conversational systems. The first study analyses over 15,000 prompts submitted to Google Gemini, revealing how users formulate structured, often imperative inputs that go well beyond traditional informational, navigational, transactional search intents. This analysis highlights the expanding role of LLMs in supporting complex tasks such as content creation and information extraction. The second study examines over 600,000 interactions with Google Assistant across 173 users, offering insight into voice-based conversational systems’ everyday utility and limitations. The data reveal a predominance of simple instructions and a lack of deeper information-seeking behaviours. Together, these studies offer a nuanced account of user intent, interaction styles, and the evolving role of conversational systems in supporting diverse and situated information needs.
Bio: Dr. Johanne Trippas is a Vice-Chancellor’s Senior Research Fellow at the School of Computing Technologies, STEM College. She works at the intersection of conversational systems, interactive information retrieval, human-computer interaction, and dialogue analysis. She has extensive experience analysing human information-seeking behaviour and developing novel approaches to personalised intelligent assistance, data-driven modelling, and profiling human behaviours. Trippas employs many research methods and consistently adopts user-centric data capture and analysis approaches, focusing on modelling and profiling human behaviours. Recently, her work has focused on developing next-generation capabilities for intelligent systems, including spoken conversational search, digital assistants in cockpits, and artificial intelligence to identify cardiac arrests. Her research aims to improve information accessibility through conversational systems, interactive information retrieval, and human-computer interaction. Trippas is particularly interested in how conversational systems can revolutionise information seeking, especially through generative interactive information retrieval and novel interfaces beyond traditional text search.







