FlowMind: Automatic Workflow Generation with LLMs

The rapidly evolving field of Robotic Process Automation (RPA) has made significant strides in automating repetitive processes, yet its effectiveness diminishes in scenarios requiring spontaneous or unpredictable tasks demanded by users. This paper introduces a novel approach, FlowMind, leveraging the capabilities of Large Language Models (LLMs) such as Generative Pretrained Transformer (GPT), to address this limitation and create an automatic workflow generation system. In FlowMind, we propose a generic prompt recipe for a lecture that helps ground LLM reasoning with reliable Application Programming Interfaces (APIs). With this, FlowMind not only mitigates the common issue of hallucinations in LLMs, but also eliminates direct interaction between LLMs and proprietary data or code, thus ensuring the integrity and confidentiality of information — a cornerstone in financial services. FlowMind further simplifies user interaction by presenting high-level descriptions of auto-generated workflows, enabling users to inspect and provide feedback effectively. We also introduce NCEN-QA, a new dataset in finance for benchmarking question-answering tasks from N-CEN reports on funds.

Introduction. The paradigm of Robotic Process Automation (RPA) has vastly transformed the landscape of task execution by enabling the automation of repetitive processes. However, its reliance on expert knowledge and well-defined procedures can fall short in situations where more spontaneous or unpredictable tasks arise. To address these challenges, we explore the capabilities of Large Language Models (LLMs) and present a framework that leverages their potential to create a versatile, dynamic, and secure workflow generation system. In this paper, we introduce a novel approach, FlowMind, that enables automatic workflow generation using LLMs, following the proposed generic lecture recipe of prompt design. Through an extensive and rigorous study, we dissect each component of our prompt design to demonstrate its importance and contributions to the overall effectiveness of automatic workflow generation.

Discussion / Conclusion. In conclusion, our work presents a significant leap in using LLMs for automatic workflow generation. By combining lecture prompt design, user feedback, and secure, grounded reasoning, FlowMind provides a reliable, adaptable, and efficient solution for handling spontaneous tasks with auto-genenated workflows. Our work opens new avenues for more widespread adoption of LLMs, particularly in industries where data security and the spontaneity of tasks are of paramount importance. We also introduce a new finance dataset NCEN-QA, which serves as a robust benchmark platform for automatic workflow generation systems on N-CEN reports question-answering tasks about funds, thus providing a valuable resource for the broader research community. In the future, it’s worth investigating crowdsourcing user feedback to refine workflows in FlowMind at scale, as well as life-long learning over past user-approved examples to evolve its performance over time. In addition, FlowMind can be expanded in the future to handle big libraries of APIs by retrieving the most relevant APIs for a given task given embedding similarity.