Saturday 15 March 2025
The human brain has long been known for its impressive ability to reason and make connections between seemingly unrelated pieces of information. From recognizing patterns in a complex sequence of events to understanding the relationships between abstract concepts, our brains are capable of performing feats that were once thought to be exclusive to computers.
Recently, researchers have been working to develop artificial intelligence (AI) systems that can mimic this ability to reason and make connections. One of the most promising approaches is based on a type of AI called transformers, which use self-attention mechanisms to process input data and generate outputs.
In a new study, scientists have demonstrated that transformers are capable of performing complex compositional reasoning tasks, which involve integrating knowledge from multiple sources to form a coherent understanding of a given situation. This ability is similar to the way humans reason and make connections between different pieces of information.
The researchers trained the transformer models on a dataset called FTCT (Fragmented at Training, Chained at Testing), which consisted of causal graphs with multiple vertices and edges. The training data was designed to simulate real-world scenarios where knowledge is fragmented across different sources, and the testing data included longer causal chains that required the model to integrate information from multiple sources.
The results were impressive: the transformer models were able to perform complex compositional reasoning tasks, including generating complete causal graphs and predicting missing values in incomplete graphs. The models also showed a remarkable ability to generalize to new situations, even when the training data did not include examples of the specific task being tested.
One of the most interesting findings was that the transformers’ performance improved as the number of shots increased, but only up to a point. After that, the performance actually decreased. This suggests that there may be an optimal amount of information that needs to be presented to the model in order for it to learn effectively, and that too much or too little information can actually hinder its ability to reason.
The researchers also explored the limitations of the transformer models when it comes to reasoning with incomplete intermediate steps. They found that while the models were able to perform well on complete causal graphs, they struggled when faced with incomplete graphs. This suggests that the models may need further training or fine-tuning in order to effectively handle real-world scenarios where information is missing or unclear.
Despite these limitations, the study demonstrates the significant potential of transformer-based AI systems for complex compositional reasoning tasks.
Cite this article: “Transformers Show Promise in Complex Compositional Reasoning Tasks”, The Science Archive, 2025.
Ai, Transformers, Self-Attention Mechanisms, Compositional Reasoning, Causal Graphs, Complex Sequences, Abstract Concepts, Artificial Intelligence, Machine Learning, Cognitive Abilities
