Beyond Conversations: Spatially-Anchored Previews for Intent Disambiguation in LLM-Assisted Geometry Editing in Virtual Reality

User intent disambiguation remains a key challenge in intelligent interactive systems. While they have been widely studied in dialogue systems in 2D interfaces, research on how intent disambiguation could be incorporated within Large Language Model (LLM) assisted editing workflows in immersive environments remains limited. Recent advances in LLMs create opportunities to leverage the immersive nature of virtual and augmented reality (VR/AR) environments to provide better disambiguation support. In this paper, we evaluate how traditional dialogue-based disambiguation can be augmented with spatially-anchored graphical previews to resolve ambiguous user commands in LLM-assisted parameter-driven editing workflows. A within-subjects study in which 24 participants completed complex geometry editing tasks in VR simulate scenarios where VR scenes are controlled by numerical parameters. Compared with the condition where disambiguation is not available, quantitative metrics and qualitative feedback indicate that a hybrid approach which combines clarification questions and graphical previews can support better interaction stability with fewer conversation rounds while improving user experience.

Introduction. Disambiguating user intent is crucial in interactive systems, especially in complex 3D spatial domains. Recent works have demonstrated the possibility of incorporating large language models (LLMs) in 3D environments and the potential benefits [8, 12, 18, 41, 45]. However, an unaddressed challenge lies in limited user agency due to ambiguous user input and the lack of mechanisms integrated in LLM-assisted scene editing workflows to improve user agency and reduce errors [8, 52]. Various non-LLM systems have explored effective disambiguation through clarification questions [1, 13] and graphical previews [30]. There is a missed opportunity to explore how these approaches could be adapted for spatial environments. In this work, graphical previews refer to a hybrid of 2D UI image previews and spatially-anchored 3D overlays. Unlike many speech-and-pointing works [4, 7, 29, 46] which study referential disambiguation, we clarify that this work does not study disambiguation on the input side of interaction.

Discussion / Conclusion. 5.2 Performance Consistency and Efficiency A key finding is that clarification questions only (CQ condition) and the combined use of clarification questions and graphical previews (CQGP condition) both significantly reduced task progression variability as evidenced by the MSSD metric, guiding users towards a more stable task progression trajectory compared with the condition without disambiguation support. The maximum closeness score did not differ significantly across conditions, and the CQGP condition required significantly fewer conversation rounds to complete the task, which suggests enhanced interaction efficiency. This finding is novel compared to 2D interfaces, where disambiguation strategies have been shown to improve task performance [30] and extends earlier work by Horvitz [22] on mixed-initiative interfaces by demonstrating that in immersive environments, text and visual feedback can guide users towards a smoother task progression trajectory.