Decision Complexity and Trust in Large Language Model Advisors
Eunsol Cho, João Sedoc, Arun Sundararajan
Abstract
As organizations increasingly deploy large language models (LLMs) as decision-support tools, individual adoption remains uneven, with trust emerging as a central barrier. We investigate how decision complexity shapes behavioral trust in LLM advisors through an incentivized investment experiment in which participants made initial allocations, received advice from an LLM (OpenAI GPT-4o), and chose whether to revise. Complexity was manipulated via the number of possible lottery outcomes (2, 4, 8, or 16), and behavioral trust was measured using weight of advice (WOA). Results show that WOA increased significantly under higher complexity, with participants in the most complex condition exhibiting a 41 percentage point increase compared to the simplest condition. However, the effect of complexity on increasing WOA was moderated by cognitive ability: it was negligible among participants with lower cognitive ability but pronounced among those with higher cognitive ability. Further analysis indicated that perceived understanding of the LLM’s reasoning declined with complexity, particularly for participants with lower cognitive ability. Taken together, these findings suggest that while LLMs can serve as especially valuable advisors in cognitively demanding contexts, their effectiveness depends on both user capabilities and how advice is perceived and understood. Our study contributes to research on trust in AI by extending behavioral trust measures to conversational agents and highlighting the contextual role of decision complexity in shaping human–AI collaboration.
- Working Paper
- Presented in Wharton AI and the Future of Work Conference (May 2025), BizAI Conference (Mar 2025)
- Online experiment on how complexity of decision problem affects behavioral trust in LLM advisor and how advisee’s cognitive ability moderates the effect.
Do Matching Titles and Thumbnails Drive Clicks? The Role of Semantic Similarity in Multimodal Video Representations
Eunsol Cho, Jaeung Sim, Daegon Cho, Jiyong Eom
Abstract
In the attention economy, the ability to capture user curiosity is critical for the success of online video platforms. A key channel through which platforms capture attention is the pre-consumption representation of videos, in which thumbnails and titles jointly shape viewer expectations of the content. This study examines how the semantic congruence between these multimodal elements influences viewing behavior. In doing so, we first analyzed a 10K+ sample of popular YouTube videos in the U.S. market to observe the associations between congruence and viewership by information modalities in the field. By measuring information congruence with shared multimodal embedding, we find that image–text similarity has a positive association with view counts, suggesting that moderate congruence best stimulates curiosity, while both low and high congruence suppress engagement. In contrast, text–text similarity is negatively associated with views, indicating that redundancy across textual modalities reduces curiosity. Robustness checks using alternative similarity measures corroborate these findings. To reveal the causal relationships and underlying mechanisms, we are finalizing the design of choice-based online experiments and are preparing for pre-registration. Our results are expected to underscore the nuanced role of multimodal design in attention capture and provide implications for both theory and practice in digital content strategy.
- Working Paper
- Presented in INFORMS Annual Meeting (Oct 2025, scheduled), CIST (Oct 2023), WITS (Dec 2021)
- Analysis on 10K+ popular YouTube videos in the U.S to reveal how semantic similarity between image and text components of thumbnail image and the title affects video views.
Redefining Objectives Under Algorithmic Ambiguity: Addressing Ambiguity Aversion through Regularization
Eunsol Cho
Abstract
Organizations increasingly deploy predictive models in settings where users differ in their tolerance for uncertainty about model performance. We argue that aligning model objectives—via regularization—can better serve users with heterogeneous ambiguity attitudes than focusing solely on inputs or mean accuracy. Using the α-Maxmin Expected Utility (α-MEU) framework, we theorize that regularization induces a trade-off between the mean and the across-dataset variance of out-of-sample errors, and that ambiguity-averse users will favor objectives that temper variance even at some cost to the mean. We evaluate this idea with simulations on synthetic data and with controlled simulations seeded by a real-world meal-delivery dataset. In both settings, we observe a pronounced mean–variance trade-off as regularization changes. With synthetic data, the α-MEU objective selects different regularization levels across ambiguity attitudes, consistent with the theory. In contrast, in the real-data setting the α-MEU objective becomes monotone in regularization, yielding the same choice across attitudes because the weighting of worst- and best-case errors moves counter to the mean. These findings suggest that user-centric objective tuning can be a useful design lever for predictive models and explore conditions under which ambiguity attitudes are more or less likely to shape model preferences.
- Working Paper
- Simulation using both synthetic data and a real-world food delivery dataset to examine how tuning regularization can align model objectives with users’ ambiguity attitudes.
The Impact of Large Language Models on Consumer Bias in Gym Membership Decisions
Eunsol Cho, Sagit Bar-Gill, João Sedoc
Abstract
Consumers systematically overestimate their future gym attendance, leading to biased gym membership choices and welfare losses. While prior research has focused on post-purchase interventions to mitigate present bias in gym usage, less is known about how to reduce projection bias at the point of purchase. We study whether interaction with large language models (LLMs) can improve expectation formation and contract choice in this setting. In a field experiment with NYU Athletics, participants choose between an annual membership and a 10-visit pass bundle. Control conditions reflect best-practice disclosures, including pricing information, historical usage averages, and threshold calculations. Treatment participants consult with an LLM provided with either pricing alone or pricing plus historical averages. Preliminary results show that both historical-average disclosure and interaction with an LLM given only pricing information significantly increase pass-bundle choice. However, when the LLM is also given historical averages, its effectiveness disappears. Analysis of chat transcripts suggests this occurs because the LLM often uses historical averages to reinforce, rather than correct, consumers’ optimistic usage expectations. Ongoing work links contract choice to realized gym visits to assess welfare implications and mechanisms driving choice shifts.
- Work-in-progress
- Field experiment in collaboration with NYU Athletics examining whether LLM advice on gym membership choices can mitigate consumer biases driven by cognitive limitations and projection bias.
Eunsol Cho; 