SymAgent: A Neural-Symbolic Self-Learning Agent Framework for Complex Reasoning over Knowledge Graphs

Recent advancements have highlighted that Large Language Models (LLMs) are prone to hallucinations when solving complex reasoning problems, leading to erroneous results. To tackle this issue, researchers incorporate Knowledge Graphs (KGs) to improve the reasoning ability of LLMs. However, existing methods face two limitations: 1) they typically assume that all answers to the questions are contained in KGs, neglecting the incompleteness issue of KGs, and 2) they treat the KG as a static repository and overlook the implicit logical reasoning structures inherent in KGs. In this paper, we introduce SymAgent, an innovative neural-symbolic agent framework that achieves collaborative augmentation between KGs and LLMs. We conceptualize KGs as dynamic environments and transform complex reasoning tasks into a multi-step interactive process, enabling KGs to participate deeply in the reasoning process. SymAgent consists of two modules: Agent-Planner and Agent- Executor. The Agent-Planner leverages LLM’s inductive reasoning capability to extract symbolic rules from KGs, guiding efficient question decomposition. The Agent-Executor autonomously invokes predefined action tools to integrate information from KGs and external documents, addressing the issues of KG incompleteness. Furthermore, we design a self-learning framework comprising online exploration and offline iterative policy updating phases, enabling the agent to automatically synthesize reasoning trajectories and improve performance.

For instance, in Figure 1, given the question Where was the person who recorded "I’m Gonna Get Drunk and Play Hank Williams" born?, the symbolic rule 𝑓 𝑒𝑎𝑡𝑢𝑟𝑒𝑑_𝑎𝑟𝑡𝑖𝑠𝑡 .𝑟𝑒𝑐𝑜𝑟𝑑𝑖𝑛𝑔𝑠 (𝑒1, 𝑒2)∧𝑝𝑒𝑟𝑠𝑜𝑛.𝑝𝑙𝑎𝑐𝑒 _𝑜 𝑓 _𝑏𝑖𝑟𝑡ℎ(𝑒2, 𝑒3) derived from the KG serves as an abstract representation of the question, revealing the intrinsic connection between question decomposition and KG structural patterns. In contrast, retrieval-based methods often suffer from superficial correlations, retrieving semantically similar but irrelevant information and even harmful disturbance, leading to degraded model performance. Moreover, both methods typically assume that all factual triples required for each question are entirely covered by the KG, which is unrealistic for manually curated KGs. When KGs fail to cover the necessary information, parser-based methods struggle to execute SPARQL queries effectively, limiting their ability to provide accurate answers or engage in complex reasoning tasks.

In light of these limitations, we delve into the exploration of the effective fusion of KGs and LLMs, enabling their collective augmentation in complex reasoning tasks. Fundamentally, realizing this integration poses several significant challenges: (i) Semantic Gap. Enabling the KG to participate deeply in the reasoning process of LLM requires aligning the symbolic structure of KGs with the neural representations of LLMs. (ii) Incompleteness of KG. When encountering insufficient information, it is necessary to retrieve relevant unstructured documents and identify missing triples consistent with the KG’s semantic granularity during the reasoning process. (iii) Learning with Limited Supervision. The complexity of tasks and the current limitation of having only natural language input-output pairs make it difficult to unlock the full reasoning potential of LLMs.

By treating the KG as a dynamic environment, we transform complex reasoning tasks into multistep interactive processes, enabling in-depth analysis and proper decomposition of complex questions. Specifically, SymAgent comprises two key components: a planning module and an execution module. The planning module leverages LLM’s inductive reasoning to derive symbolic rules from the KG, creating high-level plans for aligning natural language questions with the KG structure and employing it as a navigational tool. In the execution module, we extend the agent’s capacity by curating a multi-functional toolbox, enabling the manipulation of both structured data and unstructured documents. By engaging in a thought-action-observation loop, the agent continuously reflects on the derived plan, action execution results, and past interactions to autonomously orchestrate action tools. This process not only allows for the collection of the necessary information to answer the question but also simultaneously identifies missing factual triples to complete the KG, addressing the challenge of KG incompleteness. Given the lack of well-annotated expert trajectories, we introduce a self-learning framework, which includes online exploration and offline iterative policy updates. Through continuous interaction with the KG environment, the agent can self-synthesize and refine trajectory data without human annotation, empowering performance improvement.