Abductive Reasoning with the GPT-4 Language Model: Case studies from criminal investigation, medical practice, scientific research

By Remo Pareschi, Stake Lab, University of Molise, Campobasso, Italy

https://arxiv.org/abs/2307.10250

“We favor a dialogical approach for several reasons. Firstly, it aligns with the established methodology of Knowledge Elicitation (KE), developed in the 1980s during the advent of Expert Systems. Based on interview techniques to extract knowledge from domain experts (Hoffman et al., 1995), KE is still applicable today for verifying knowledge stored in LLMs like GPT- 4. By adopting KE techniques, we assess performance through in-depth interviews on selected topics, providing a more comprehensive assessment of the model's capabilities.

Moreover, Abduction, the form of reasoning we focus on, naturally fits into a dialogical model where one dialoguer provides an explanation, and the other challenges it (Walton, 2004). Lastly, our interview approach aligns with a vision of human-AI interaction as a collaborative endeavor. This perspective envisions augmented intelligence, where human and artificial intelligence integrate seamlessly, enhancing effectiveness and creativity.” 

In this extended dialogue, we could delve deeper into the AI's capabilities. This included its ability to formulate and respond to questions about abduction, identify cause and effect relationships, use deductive logic to validate abductive explanations, and navigate the interplay between deductive, inductive, and abductive reasoning. The AI also showcased its capacity to handle theories that use future predictions to explain both present and future phenomena. This case study comprehensively explores the potential and challenges of employing abductive reasoning in a scientific context. It also highlights the AI's meta-reasoning on abduction, which provides self-awareness about the assumptions used during hypothesis formulation and evaluation. This capability, which the AI demonstrated effectively, can be a valuable contribution to a research team. Moreover, ChatGPT proactively and courteously corrected its interlocutor's mistake regarding the timeline of the decline of one of the theories discussed (the Steady State Theory), showcasing its active engagement in the conversation. Lastly, the AI conducted a self-assessment.”