Saturday 01 February 2025
Autonomous driving has been a topic of intense research and development in recent years, with many companies and institutions working towards creating self-driving cars that can navigate roads safely and efficiently. However, one of the biggest challenges in achieving this goal is developing an AI system that can understand and respond to complex driving scenarios.
A team of researchers has made significant progress in addressing this challenge by introducing a new framework called PKRD-CoT, which stands for Prompt Knowledge Reasoning Decision-making Chain-of-Thought. This framework combines four essential capabilities – perception, knowledge, reasoning, and decision-making – to enable large language models (LLMs) to make informed decisions while driving.
The researchers tested six different LLMs, including GPT-4, Claude, LLava1.6, Qwen-VL-Plus, CogVLM chat, and Minigpt4, using a series of experiments designed to evaluate their ability to perform these four capabilities. The results showed that GPT-4 performed the best overall, with Claude and LLava1.6 close behind.
One of the key findings was that LLMs can be trained to recognize and respond to complex driving scenarios, such as recognizing traffic lights, pedestrians, and other vehicles. However, the models also showed limitations in certain areas, such as mathematically calculating distances between vehicles.
The researchers used a combination of real-world images and simulated driving scenarios to test the LLMs’ abilities. They found that GPT-4 was able to correctly recognize and describe objects in the image, such as cars and pedestrians, while Claude and LLava1.6 performed well in recognizing traffic lights and pedestrian crossings.
In terms of decision-making, the researchers used a scenario where the models had to decide whether to stop or continue driving at a red light. GPT-4 took into account factors such as the distance between vehicles and the speed of oncoming traffic before making a decision, while Qwen-VL-Plus prioritized smooth speed adjustments for safety and passenger comfort.
The study’s findings have significant implications for the development of autonomous driving systems. The researchers believe that their framework can be used to create more advanced AI systems that can better understand and respond to complex driving scenarios. This could ultimately lead to safer and more efficient self-driving cars that are capable of navigating a wide range of environments.
Overall, this study demonstrates the potential of large language models in autonomous driving applications.
Cite this article: “Advances in AI-Driven Autonomous Driving: A Framework for Complex Scenario Recognition and Decision-Making”, The Science Archive, 2025.
Autonomous, Driving, Ai, Language Models, Pkrd-Cot, Framework, Decision-Making, Reasoning, Perception, Knowledge
