Defending Quality Metrics Against Adversarial Attacks
Benchmarking and developing empirical defenses for image quality metrics, including adversarial purification and adversarial training.
After showing that modern quality metrics are easy to manipulate, we studied how to protect them. We investigated whether adversarial purification methods from image classification transfer to quality assessment and proposed new purification methods (Gushchin et al., 2024), and studied adversarial training for image quality assessment models (Chistyakova et al., 2024).
The results were summarized in the first benchmark of defenses for image quality metrics, systematically evaluating 25 defense strategies — adversarial purification, adversarial training and certified robustness methods — against a wide range of attacks (Gushchin et al., 2025).
References
2025
ICML
Guardians of Image Quality: Benchmarking Defenses Against Adversarial Attacks on Image Quality Metrics
Alexander Gushchin, Khaled Abud, Georgii Bychkov, Ekaterina Shumitskaya, Anna Chistyakova, Sergey Lavrushkin, Bader Rasheed, Kirill Malyshev, Dmitriy Vatolin, and Anastasia Antsiferova
In Forty-second International Conference on Machine Learning, 2025
In the field of Image Quality Assessment (IQA), the adversarial robustness of the metrics poses a critical concern. This paper presents a comprehensive benchmarking study of various defense mechanisms in response to the rise in adversarial attacks on IQA. We systematically evaluate 25 defense strategies, including adversarial purification, adversarial training, and certified robustness methods. We applied 14 adversarial attack algorithms of various types in both non-adaptive and adaptive settings and tested these defenses against them. We analyze the differences between defenses and their applicability to IQA tasks, considering that they should preserve IQA scores and image quality. The proposed benchmark aims to guide future developments and accepts submissions of new methods, with the latest results available online: https://videoprocessing.ai/benchmarks/iqa-defenses.html.
@inproceedings{antsiferovaGuardians2024,title={{Guardians of Image Quality: Benchmarking Defenses Against Adversarial Attacks on Image Quality Metrics}},author={Gushchin, Alexander and Abud, Khaled and Bychkov, Georgii and Shumitskaya, Ekaterina and Chistyakova, Anna and Lavrushkin, Sergey and Rasheed, Bader and Malyshev, Kirill and Vatolin, Dmitriy and Antsiferova, Anastasia},year={2025},booktitle={Forty-second International Conference on Machine Learning},}
2024
arXiv
Adversarial purification for no-reference image-quality metrics: applicability study and new methods
Aleksandr Gushchin, Anna Chistyakova, Vladislav Minashkin, Anastasia Antsiferova, and Dmitriy Vatolin
Recently, the area of adversarial attacks on image quality metrics has begun to be explored, whereas the area of defences remains under-researched. In this study, we aim to cover that case and check the transferability of adversarial purification defences from image classifiers to IQA methods. In this paper, we apply several widespread attacks on IQA models and examine the success of the defences against them. The purification methodologies covered different preprocessing techniques, including geometrical transformations, compression, denoising, and modern neural network-based methods. Also, we address the challenge of assessing the efficacy of a defensive methodology by proposing ways to estimate output visual quality and the success of neutralizing attacks. Defences were tested against attack on three IQA metrics – Linearity, MetaIQA and SPAQ. The code for attacks and defences is available at: (link is hidden for a blind review).
@misc{vatolinAdversarial2024,title={{Adversarial purification for no-reference image-quality metrics: applicability study and new methods}},author={Gushchin, Aleksandr and Chistyakova, Anna and Minashkin, Vladislav and Antsiferova, Anastasia and Vatolin, Dmitriy},year={2024},journal={arXiv preprint arXiv:2404.06957},}
Technologies
Increasing the robustness of image quality assessment models through adversarial training
Anna Chistyakova, Anastasia Antsiferova, Maksim Khrebtov, Sergey Lavrushkin, Konstantin Arkhipenko, Dmitriy Vatolin, and Denis Turdakov
The adversarial robustness of image quality assessment (IQA) models to adversarial attacks is emerging as a critical issue. Adversarial training has been widely used to improve the robustness of neural networks to adversarial attacks, but little in-depth research has examined adversarial training as a way to improve IQA model robustness. This study introduces an enhanced adversarial training approach tailored to IQA models; it adjusts the perceptual quality scores of adversarial images during training to enhance the correlation between an IQA model’s quality and the subjective quality scores. We also propose a new method for comparing IQA model robustness by measuring the Integral Robustness Score; this method evaluates the IQA model resistance to a set of adversarial perturbations with different magnitudes. We used our adversarial training approach to increase the robustness of five IQA models. Additionally, we tested the robustness of adversarially trained IQA models to 16 adversarial attacks and conducted an empirical probabilistic estimation of this feature.
@article{turdakovIncreasing2024,title={{Increasing the robustness of image quality assessment models through adversarial training}},author={Chistyakova, Anna and Antsiferova, Anastasia and Khrebtov, Maksim and Lavrushkin, Sergey and Arkhipenko, Konstantin and Vatolin, Dmitriy and Turdakov, Denis},year={2024},journal={Technologies},}