Boosting Logical Reasoning in Large Language Models through a New Framework: The Graph of Thought
“Consequently, diverging from the previously established works, this paper introduces a distinct problem-solving approach.
It mainly contributes in three aspects:
Introduction of a novel graph structure to enhance the connections between different lines of thought (nodes).
Implementation of a checking mechanism to ensure the accuracy of the connections between different lines of thought.
Proposal of a new graph updating method, designed to facilitate rapid iteration for further reasoning.
Graph of Thought
Our design contains three key innovations:
Graph Structure: By constructing thought graphs, we model various aspects and concepts of the problem as nodes, while their relationships and connections are represented by edges. This graphical representation facilitates the LLM in capturing and comprehending complex logical relationships, thereby improving its reasoning capabilities and accuracy in answering questions.
Inspection Mechanism: We address the challenge of reducing errors in LLMs by implementing a rechecking process for potentially correct results. In this process, we provide a more robust estimation of result accuracy. By calculating the confidence or probability of different candidate answers, we can effectively assess and weigh their reliability. This allows us to select the final answer based on the probabilities associated with each candidate result. Our re-evaluation mechanism is executed using our Checker function, which offers greater precision compared to traditional scoring or voting systems. This enhanced accuracy arises primarily because: i) we filter for a single, optimal result rather than selecting several decent outcomes, and ii) our checker function comprises multiple linked inspectors, ensuring a more rigorous review.
Convenient Graph Updating: Throughout the graph traversal process, thoughts that pass the Checker function are continuously added to our condition sequence. For complex problems that may not be resolved in a single graph traversal, there’s no need to keep track of all previously traversed paths. We can simply re-input our updated condition sequence. This significantly reduces potential redundant reasoning. A prime example of this can be seen in our experimental section’s third task, with the graph update process illustrated in Figure 5.”