Sunday 30 March 2025
The paper under review presents a novel approach to improving the transferability of adversarial examples in multimodal large language models (MLLMs). The authors demonstrate that by dynamically perturbing the vision-language alignment in these models, they can significantly enhance the transferability of adversarial attacks across different MLLMs.
Multimodal large language models have become increasingly popular in recent years due to their ability to process and generate text, as well as understand and generate visual content. However, these models are not immune to adversarial attacks, which can be used to manipulate their output and potentially deceive users. The transferability of adversarial examples – the ability to use an attack generated for one model on another model – is a critical concern in this area.
The authors’ approach, dubbed Dynamic Vision-Language Alignment (DynVLA), involves injecting dynamic perturbations into the vision-language connector of MLLMs during training. This allows the models to learn more robust representations that are less susceptible to adversarial attacks. The authors demonstrate that DynVLA can improve the transferability of adversarial examples across different MLLMs, including state-of-the-art models such as InternVL and Gemini.
The key innovation of DynVLA is its ability to adapt to different vision-language alignment patterns in each model. By dynamically perturbing these alignments during training, the authors show that they can improve the robustness of the models against adversarial attacks. This approach is particularly effective when used in combination with other techniques for improving robustness, such as data augmentation and regularization.
The paper presents a range of experimental results demonstrating the effectiveness of DynVLA. The authors show that their approach can significantly improve the transferability of adversarial examples across different MLLMs, while also reducing the accuracy of these models on clean test data. They also demonstrate that DynVLA can be used to improve the robustness of MLLMs against a range of different types of attacks.
Overall, this paper presents an important contribution to the field of multimodal large language models and adversarial attacks. The authors’ approach offers a promising new direction for improving the robustness of these models against adversarial threats, and has significant implications for their use in a wide range of applications, from natural language processing to computer vision.
Cite this article: “Improving Transferability of Adversarial Attacks on Multimodal Large Language Models through Dynamic Vision-Language Alignment”, The Science Archive, 2025.
Multimodal Large Language Models, Adversarial Attacks, Transferability, Robustness, Vision-Language Alignment, Dynamic Perturbations, Data Augmentation, Regularization, Natural Language Processing, Computer Vision
