Skip to content

Commit 739a312

Browse files
sarathknvsarathknv
committed
add semantic attacks
1 parent baaffd7 commit 739a312

File tree

226 files changed

+1842
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

226 files changed

+1842
-0
lines changed

‎.gitignore

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
2+
.DS_Store
3+
*.pyc

‎semantic_adversarial_examples/.idea/.gitignore

+3
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎semantic_adversarial_examples/.idea/inspectionProfiles/profiles_settings.xml

+6
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎semantic_adversarial_examples/.idea/misc.xml

+4
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎semantic_adversarial_examples/.idea/modules.xml

+8
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎semantic_adversarial_examples/.idea/semantic_adversarial_examples.iml

+8
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎semantic_adversarial_examples/.idea/vcs.xml

+6
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎semantic_adversarial_examples/README.md

+125
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
# Semantic Adversarial Examples
2+
3+
>Images that are arbitrarily perturbed to fool the model, but in such a way that the modified image semantically represents the same object as the original image.
4+
>
5+
> &mdash; <cite>Semantic Adversarial Examples, Hosseini et al., CVPRW 2018.</cite>
6+
7+
These attacks are discussed in the following papers:
8+
1. [Semantic Adversarial Examples](https://arxiv.org/pdf/1804.00499.pdf) (CVPR Workshops 2018)
9+
2. [Towards Compositional Adversarial Robustness: Generalizing Adversarial Training to Composite Semantic Perturbations](https://arxiv.org/pdf/2202.04235.pdf) (CVPR 2023)
10+
11+
This repository implements **only single semantic attacks** discussed in [2]. The attacks are
12+
constructed using Projected Gradient Descent (PGD) on the following components:
13+
* Hue
14+
* Saturation
15+
* Rotation
16+
* Brightness
17+
* Contrast
18+
19+
See section 3.2. of [2] for the math behind these attacks.
20+
21+
22+
## Dependencies
23+
* Python3
24+
* PyTorch
25+
* [Kornia](https://github.com/kornia/kornia) (A differentiable computer vision library for PyTorch)
26+
* OpenCV
27+
* NumPy
28+
* tqdm
29+
30+
## Contents
31+
32+
`attacks.py`: contains both gradient-based search and random search of the five semantic perturbation parameters.
33+
`main.py`: computes the robust accuracy of a model against single attacks. Need to manually change the model and the attack.
34+
`save_examples.py`: saves some adversarial images to disk, along with their original images.
35+
`examples`: contains some adversarial examples.
36+
`models`: VGG16 and ResNet models.
37+
`weights`: VGG16 and ResNet50 weights, trained on CIFAR-10.
38+
`visualizations`: codes to visualize the attacks.
39+
40+
41+
42+
## Experiments
43+
44+
Robust accuracy of single semantic attacks on ResNet50 trained on CIFAR-10.
45+
46+
| | Clean | Hue | Saturation | Rotation | Brightness | Contrast |
47+
|:------|------:|:-----------:|:-----------:|:-----------:|:-----------:|:-----------:|
48+
| Mine | 92.72 | 81.65 | 92.37 | 88.49 | 90.04 | 91.40 |
49+
| Paper | 95.2 | 81.8 ± 0.0 | 94.0 ± 0.0 | 88.1 ± 0.1 | 92.1 ± 0.1 | 93.7 ± 0.1 |
50+
51+
52+
53+
## Examples
54+
Untargeted examples (read ibm repo different nomenclature)
55+
56+
### Hue
57+
58+
| Clean | <img src="examples/hue/6_y_1_y_pred_1.png" alt="img" width="128"> | <img src="examples/hue/15_y_8_y_pred_8.png" alt="img" width="128"> | <img src="examples/hue/20_y_7_y_pred_7.png" alt="img" width="128"> | <img src="examples/hue/24_y_5_y_pred_5.png" alt="img" width="128"> | <img src="examples/hue/25_y_2_y_pred_2.png" alt="img" width="128"> |
59+
|:------------------------:|:--------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------:|
60+
| **Perturbed** | <img src="examples/hue/6_y_1_y_pred_1_y_adv_5_factor_1.836.png" alt="img" width="128"> | <img src="examples/hue/15_y_8_y_pred_8_y_adv_6_factor_-0.193.png" alt="img" width="128"> | <img src="examples/hue/20_y_7_y_pred_7_y_adv_9_factor_-1.913.png" alt="img" width="128"> | <img src="examples/hue/24_y_5_y_pred_5_y_adv_4_factor_3.142.png" alt="img" width="128"> | <img src="examples/hue/25_y_2_y_pred_2_y_adv_5_factor_2.274.png" alt="img" width="128"> |
61+
| **Pred before** | automobile | ship | horse | dog | bird |
62+
| **Pred after**<br/>(hue) | dog<br/>(1.836) | frog<br/>(-0.193) | truck<br/>(-1.913) | deer<br/>(3.142) | dog<br/>(2.274) |
63+
64+
65+
66+
### Saturation
67+
68+
| Clean | <img src="examples/saturation/213_y_9_y_pred_9.png" alt="img" width="128"> | <img src="examples/saturation/412_y_3_y_pred_3.png" alt="img" width="128"> | <img src="examples/saturation/434_y_3_y_pred_3.png" alt="img" width="128"> | <img src="examples/saturation/468_y_7_y_pred_7.png" alt="img" width="128"> | <img src="examples/saturation/776_y_2_y_pred_2.png" alt="img" width="128"> |
69+
|:-------------------------------:|:-----------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------:|
70+
| **Perturbed** | <img src="examples/saturation/213_y_9_y_pred_9_y_adv_8_factor_0.700.png" alt="img" width="128"> | <img src="examples/saturation/412_y_3_y_pred_3_y_adv_5_factor_1.156.png" alt="img" width="128"> | <img src="examples/saturation/434_y_3_y_pred_3_y_adv_1_factor_0.700.png" alt="img" width="128"> | <img src="examples/saturation/468_y_7_y_pred_7_y_adv_2_factor_1.159.png" alt="img" width="128"> | <img src="examples/saturation/776_y_2_y_pred_2_y_adv_5_factor_0.874.png" alt="img" width="128"> |
71+
| **Pred before** | truck | cat | cat | horse | bird |
72+
| **Pred after**<br/>(saturation) | ship<br/>(0.700) | dog<br/>(1.156) | automobile<br/>(0.700) | bird<br/>(1.159) | dog<br/>(0.874) |
73+
74+
75+
76+
### Rotation
77+
78+
| Clean | <img src="examples/rotation/2_y_8_y_pred_8.png" alt="img" width="128"> | <img src="examples/rotation/9_y_1_y_pred_1.png" alt="img" width="128"> | <img src="examples/rotation/15_y_8_y_pred_8.png" alt="img" width="128"> | <img src="examples/rotation/57_y_7_y_pred_7.png" alt="img" width="128"> | <img src="examples/rotation/70_y_2_y_pred_2.png" alt="img" width="128"> |
79+
|:-----------------------------:|:--------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------------:|
80+
| **Perturbed** | <img src="examples/rotation/2_y_8_y_pred_8_y_adv_1_factor_-6.318.png" alt="img" width="128"> | <img src="examples/rotation/9_y_1_y_pred_1_y_adv_3_factor_9.400.png" alt="img" width="128"> | <img src="examples/rotation/15_y_8_y_pred_8_y_adv_6_factor_0.383.png" alt="img" width="128"> | <img src="examples/rotation/57_y_7_y_pred_7_y_adv_3_factor_-9.586.png" alt="img" width="128"> | <img src="examples/rotation/70_y_2_y_pred_2_y_adv_3_factor_10.000.png" alt="img" width="128"> |
81+
| **Pred before** | ship | automobile | ship | horse | bird |
82+
| **Pred after**<br/>(rotation) | automobile<br/>(-6.318) | cat<br/>(9.400) | frog<br/>(0.383) | cat<br/>(-9.586) | cat<br/>(10.000) |
83+
84+
85+
86+
### Brightness
87+
88+
| Clean | <img src="examples/brightness/2_y_8_y_pred_8.png" alt="img" width="128"> | <img src="examples/brightness/57_y_7_y_pred_7.png" alt="img" width="128"> | <img src="examples/brightness/103_y_3_y_pred_3.png" alt="img" width="128"> | <img src="examples/brightness/118_y_2_y_pred_2.png" alt="img" width="128"> | <img src="examples/brightness/313_y_0_y_pred_0.png" alt="img" width="128"> |
89+
|:-------------------------------:|:----------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------:|
90+
| **Perturbed** | <img src="examples/brightness/2_y_8_y_pred_8_y_adv_1_factor_-0.200.png" alt="img" width="128"> | <img src="examples/brightness/57_y_7_y_pred_7_y_adv_5_factor_-0.150.png" alt="img" width="128"> | <img src="examples/brightness/103_y_3_y_pred_3_y_adv_6_factor_0.125.png" alt="img" width="128"> | <img src="examples/brightness/118_y_2_y_pred_2_y_adv_6_factor_-0.134.png" alt="img" width="128"> | <img src="examples/brightness/313_y_0_y_pred_0_y_adv_2_factor_-0.023.png" alt="img" width="128"> |
91+
| **Pred before** | ship | horse | cat | bird | airplane |
92+
| **Pred after**<br/>(brightness) | automobile<br/>(-0.200) | dog<br/>(-0.150) | frog<br/>(0.125) | frog<br/>(-0.134) | bird<br/>(-0.023) |
93+
94+
95+
96+
### Contrast
97+
98+
| Clean | <img src="examples/contrast/2_y_8_y_pred_8.png" alt="img" width="128"> | <img src="examples/contrast/57_y_7_y_pred_7.png" alt="img" width="128"> | <img src="examples/contrast/70_y_2_y_pred_2.png" alt="img" width="128"> | <img src="examples/contrast/103_y_3_y_pred_3.png" alt="img" width="128"> | <img src="examples/contrast/140_y_6_y_pred_6.png" alt="img" width="128"> |
99+
|:-----------------------------:|:-------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------------:|:---------------------------------------------------------------------------------------------:|
100+
| **Perturbed** | <img src="examples/contrast/2_y_8_y_pred_8_y_adv_1_factor_0.773.png" alt="img" width="128"> | <img src="examples/contrast/57_y_7_y_pred_7_y_adv_3_factor_0.836.png" alt="img" width="128"> | <img src="examples/contrast/70_y_2_y_pred_2_y_adv_3_factor_1.109.png" alt="img" width="128"> | <img src="examples/contrast/103_y_3_y_pred_3_y_adv_6_factor_0.700.png" alt="img" width="128"> | <img src="examples/contrast/140_y_6_y_pred_6_y_adv_3_factor_0.711.png" alt="img" width="128"> |
101+
| **Pred before** | ship | horse | bird | cat | frog |
102+
| **Pred after**<br/>(contrast) | automobile<br/>(0.773) | cat<br/>(0.836) | cat<br/>(1.109) | frog<br/>(0.700) | cat<br/>(0.711) |
103+
104+
105+
106+
## Citations
107+
108+
```bibtex
109+
@inproceedings{hosseini2018semantic,
110+
title={Semantic adversarial examples},
111+
author={Hosseini, Hossein and Poovendran, Radha},
112+
booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops},
113+
pages={1614--1619},
114+
year={2018}
115+
}
116+
```
117+
118+
```bibtex
119+
@article{tsai2022towards,
120+
title={Towards compositional adversarial robustness: Generalizing adversarial training to composite semantic perturbations},
121+
author={Tsai, Yun-Yun and Hsiung, Lei and Chen, Pin-Yu and Ho, Tsung-Yi},
122+
journal={arXiv preprint arXiv:2202.04235},
123+
year={2022}
124+
}
125+
```

0 commit comments

Comments
 (0)