Saturday 01 February 2025
Autonomous systems, such as self-driving cars and drones, are becoming increasingly common in our daily lives. However, these systems can fail, causing harm or even loss of life. To ensure their safety, it’s essential to accurately estimate the probability of failure.
Researchers have been working on developing methods for estimating failure probabilities, but they often rely on assumptions that may not hold true in real-world scenarios. A new approach, called State-dependent Proposal Adaptive Importance Sampling (SPAIS), has been developed by a team of scientists to overcome these limitations.
The key innovation behind SPAIS is its ability to adaptively generate proposals that are tailored to the specific failure modes of an autonomous system. This is achieved through a process called Markov score ascent, which iteratively refines the proposal distribution until it accurately represents the failure distribution.
To test the effectiveness of SPAIS, the researchers evaluated it on four different validation problems: an inverted pendulum, a crosswalk scenario, aircraft collision avoidance, and F-16 ground collision avoidance. In each case, they compared SPAIS to three baseline methods: Monte Carlo estimation, cross-entropy method (CEM), and policy gradient-based importance sampling (PG-AIS).
The results showed that SPAIS significantly outperformed the baseline methods in all four scenarios. It achieved an average relative error of 4%, compared to errors ranging from 10% to 50% for the other methods.
One of the key advantages of SPAIS is its ability to capture rare failure modes, which can be difficult or impossible to detect using traditional methods. This is because SPAIS uses a state-dependent proposal distribution that adapts to the specific failure modes of an autonomous system.
The researchers also found that SPAIS is more robust than other methods in the face of limited data. This is important, as real-world autonomous systems often have limited data availability and may require estimation under uncertainty.
Overall, SPAIS represents a significant step forward in the development of reliable and accurate methods for estimating failure probabilities in autonomous systems. Its adaptability and robustness make it an attractive option for ensuring the safety of these critical systems.
Cite this article: “Accurate Failure Probability Estimation for Autonomous Systems Using SPAIS”, The Science Archive, 2025.
Autonomous Systems, Self-Driving Cars, Drones, Failure Probability, Spais, Markov Score Ascent, Proposal Distribution, Importance Sampling, Monte Carlo Estimation, Cross-Entropy Method, Policy Gradient-Based Importance Sampling
