Framing Prompts as User Stories: Effects on the Output Quality of Generative AI

Abstract

This study investigates how framing prompts as user stories affects the output quality of generative AI systems. It examines whether structuring prompts in the form commonly used in software development and human-centred design (“As a user, I would like to in order...”) improves the relevance, clarity, and contextual accuracy of generated responses. A controlled experiment was conducted with tourism-related scenarios in which a large language model (LLM) was presented with prompts in both traditional and user story formats. The outputs were evaluated using a hybrid method that combines automatic metrics (BERTScore) and human expert evaluation based on the IE-Information Extraction framework (precision, recall, F1, and error rate) to capture qualitative aspects of response quality. The results show that prompts formatted as user stories consistently yield higher-quality responses, especially in matching the user's intent and providing accurate, relevant content. These findings highlight the role of prompt structure in shaping LLM performance and suggest that user-centred prompt design can improve generative AI applications in domain-specific contexts. The study contributes to the prompt engineering literature and offers practical implications for improving human–AI interaction by formulating inputs more deliberately and structurally.