EFAG Seminar with Professor Victor Maas (University of Amsterdam)

Using Large Language Models to Explore Contextualization Effects in Economics-Based Accounting Experiments

Abstract

Most economics-based experimental accounting research tests hypotheses about the behavior of professionals in business settings. There is an ongoing debate about whether the specific business context should be made explicit in experiments, or whether researchers should instead rely on abstract, stylized settings with neutrally framed instructions. In this paper, we argue that the effects of contextualization are highly idiosyncratic and therefore extremely difficult to anticipate without empirical testing. We then conjecture that large language models (LLMs) can be of great value here. We introduce ‘botex’, a software tool that allows researchers to run pilot tests with simulated participants. We illustrate the use of botex in a series of experiments comparing neutral and contextualized versions of three widely-used experimental games. We find that contextualization has significant, but not always consistent or straightforward effects on participants’ behavior, highlighting the potential value of using LLMs in the experimental design stage.

Link to Paper

About the speaker

Victor Maas is a Professor of Accounting at the University of Amsterdam, where he earned his PhD in 2007. After a stint at Erasmus University Rotterdam, he returned to Amsterdam as a full professor in 2014. He teaches various accounting and research methods courses across different programs and focuses his research on the interplay of economic, cognitive, and social factors in organizational behavior. His research interests include discretionary performance evaluation processes and strategic and dysfunctional behavior in organizations. His work, published in top accounting journals such as the Journal of Accounting Research, The Accounting Review, Contemporary Accounting Research, and Accounting, Organizations and Society, explores these themes extensively. He is the Senior Editor of Behavioral Research in Accounting and serves on several editorial boards. He also holds an adjunct position at NHH (Bergen, Norway). He is also the coordinator of the European Network for Experimental Accounting Research (ENEAR) and the Chair of the Ethics Committee for Research in Economics and Business (EBEC) at the University of Amsterdam.