Mindstorms in Natural Language-Based Societies of Mind

The 2015 work on “learning to think” [28] proposed to connect both NNs through recurrent connections (trained by the second NN’s learning algorithm) that allow one NN to interview the other by sending sequences of queries or prompts (real-valued vectors) into it while receiving and interpreting answers (real-valued vectors) from it. An Algorithmic Information Theory (AIT) argument shows [28], [29] that it may be much easier for the controller NN to solve its task by inventing good prompts that address and extract relevant information in the other NN rather than learning the task from scratch.

To solve a given task, the various modules can chat with each other in a multimodal “mindstorm.”

Our concept of mindstorm is largely inspired by the success of sophisticated forms of communication within human societies, such as brainstorming, that may involve multiple rounds of communication to refine ideas or to find an agreement among multiple individuals. In human psychology, a large body of work exists which demonstrates that a solution found through brainstorming by a group of people is often superior to any individual solution