xDailyBench: Benchmarking LLMs on Professional Consultation for Real-Life Problems

Large language models (LLMs) are increasingly used for everyday assistance, yet existing benchmarks only partially reflect the requests users naturally make in practice. Real-world requests are often open-ended, casually specified, and context-dependent, requiring models not only to follow explicit instructions but also to infer unstated needs from user background and situational context. We introduce xDailyBench, a benchmark of 248 carefully curated tasks spanning 51 scenarios across personal life, white-collar work, learning and research, and cross-domain activities. The tasks are grounded in requests that users have actually completed or genuinely intended to accomplish with AI, and are evaluated with fine-grained binary rubrics covering both explicit and implicit requirements. We evaluate 11 frontier models under standardized agentic settings. The best models achieve a task-level score of 75.6%, while all models perform substantially worse on implicit than explicit requirements, with gaps no less than 9 percentage points. These results reveal implicit requirement inference as a persistent bottleneck for reliably satisfying real-world everyday user needs.

Introduction. Agent capabilities have become the primary focus of frontier Large Language Models (LLMs) development, shifting the focus of progress from isolated capabilities toward reliable completion of complex real-world tasks. Consequently, recent benchmarks increasingly evaluate whether models can autonomously perform end-to-end work in authentic environments across diverse professional domains, such as software engineering, scientific research, web interaction, and computer use [2, 14, 26, 33]. Rather than measuring individual reasoning or tool-use skills in isolation, these evaluations emphasize a model’s ability to coordinate planning, information gathering, tool invocation, and decision making to accomplish practical objectives [23, 35]. As frontier models continue to improve, these benchmarks have rapidly evolved toward increasingly complex, long-horizon, and realistic tasks [7, 8, 26], continuously raising the bar for evaluating agent capabilities. However, autonomous execution of professional workflows represents only one practical role of AI agents.

Discussion / Conclusion. xDailyBench requires models to gather facts from multiple sources, reconcile constraints, and produce actionable deliverables. Failures can arise at different stages of this process: some models produce seemingly complete outputs without verifying whether the proposed solution is actually feasible, while others fail to converge from information gathering to synthesis and final artifact production. By jointly examining generated artifacts, rubric-level failures, and execution traces, we identify two representative failure modes: missing feasibility checks and non-convergent evidence collection. Detailed Planning without a Feasibility Check. In planning tasks, users typically ask directly for a detailed plan without additionally requiring the model to prove that its proposal is feasible.