READ-ONLY, keep in sync with next sections, do NOT change this section.
MNIST, lr=1e-4, MNISTCNN, ItAdv train, batch 32, GTX 1070, 72s/epoch
| epoch | adv acc | clean acc |
|---|---|---|
| 1 | 0.4988 | 0.9487 |
| 2 | 0.6058 | 0.9629 |
| 3 | 0.6188 | 0.9641 |
| 4 | 0.7032 | 0.9721 |
| 5 | 0.7401 | 0.9763 |
| 6 | 0.7801 | 0.9733 |
| 10 | 0.8585 | 0.9781 |
| 15 | 0.9270 | 0.9821 |
| 20 | 0.9455 | 0.9853 |
ResNet29, [0,1], lr=1e-3, batch 128, CNN training. 25s/epoch
| epoch | val acc |
|---|---|
| 1 | 0.5877 |
| 2 | 0.6367 |
| 3 | 0.6883 |
| 4 | 0.7451 |
| 5 | 0.7561 |
| 19 | 0.8551 |
| 32 | 0.8725 |
| 36 | 0.8803 |
| 41 (5e-4) | 0.8926 |
| 47 (2.5e-4) | 0.8998 |
Resnet29, [0,1], lr=2e-3, itadv, batch 128, trained until early stop, 88 epochs. 160s/epoch. Note that 4e-4 is Madry’s starting lr.
| epoch | adv acc | clean acc |
|---|---|---|
| 1 (2e-3) | 0.2312 | 0.3425 |
| 2 | 0.2698 | 0.3758 |
| 10 | 0.3562 | 0.5401 |
| 16 | 0.3846 | 0.5739 |
| 23 | 0.3984 | 0.6170 |
| 27 (1e-3) | 0.4139 | 0.6336 |
| 33 | 0.4221 | 0.6430 |
| 37 (5e-4) | 0.4353 | 0.6755 |
| 43 | 0.4353 | 0.6863 |
| 47 (2.5e-4) | 0.4471 | 0.7003 |
| 60 | 0.4511 | 0.7047 |
| 71 (6.25e-5) | 0.4573 | 0.7167 |
| 76 (5e-5) | 0.4573 | 0.7133 |
| 81 (best) | 0.4593 | 0.7200 |
| 88 | 0.4565 | 0.7236 |
MNIST, lr=1e-4, MNISTCNN, ItAdv train, GTX 1070, 72s/epoch
| epoch | adv acc | clean acc |
|---|---|---|
| 1 | 0.4988 | 0.9487 |
| 2 | 0.6058 | 0.9629 |
| 3 | 0.6188 | 0.9641 |
| 4 | 0.7032 | 0.9721 |
| 5 | 0.7401 | 0.9763 |
| 6 | 0.7801 | 0.9733 |
| 10 | 0.8585 | 0.9781 |
| 15 | 0.9270 | 0.9821 |
| 20 | 0.9455 | 0.9853 |
- ‘advacc’: 0.4988302139037433, ‘acc’: 0.9486965240641712
- ‘advacc’: 0.6057820855614974, ‘acc’: 0.9629010695187166
- ‘advacc’: 0.6188168449197861, ‘acc’: 0.9640708556149733
- ‘advacc’: 0.7032085561497327, ‘acc’: 0.9720922459893048
- ‘advacc’: 0.7401403743315508, ‘acc’: 0.9762700534759359
- ‘advacc’: 0.7800802139037433, ‘acc’: 0.9732620320855615
- ‘advacc’: 0.8584558823529411, ‘acc’: 0.9781082887700535
- ‘advacc’: 0.9269719251336899, ‘acc’: 0.9821189839572193
- ‘advacc’: 0.9455213903743316, ‘acc’: 0.9852941176470589
Train on 54000 samples, validate on 6000 samples [54/2860]
Epoch 1/200
2019-11-10 23:11:15.671647: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2019-11-10 23:11:15.792206: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
54000/54000 [==============================] - 73s 1ms/step - loss: 1.9048 - val_loss: 1.4282
evaluating ..
{'advacc': 0.4988302139037433, 'acc': 0.9486965240641712, 'cnnacc': 0.9486965240641712, 'obliacc': 0.49649064171123}
Epoch 2/200
54000/54000 [==============================] - 72s 1ms/step - loss: 1.3646 - val_loss: 1.1387
evaluating ..
{'advacc': 0.6057820855614974, 'acc': 0.9629010695187166, 'cnnacc': 0.9629010695187166, 'obliacc': 0.6047794117647058}
Epoch 3/200
54000/54000 [==============================] - 72s 1ms/step - loss: 1.1461 - val_loss: 1.0028
evaluating ..
{'advacc': 0.6188168449197861, 'acc': 0.9640708556149733, 'cnnacc': 0.9640708556149733, 'obliacc': 0.6171457219251337}
Epoch 4/200
54000/54000 [==============================] - 72s 1ms/step - loss: 0.9889 - val_loss: 0.8356
evaluating ..
{'advacc': 0.7032085561497327, 'acc': 0.9720922459893048, 'cnnacc': 0.9720922459893048, 'obliacc': 0.7037098930481284}
Epoch 5/200
54000/54000 [==============================] - 72s 1ms/step - loss: 0.8573 - val_loss: 0.7217
evaluating ..
{'advacc': 0.7401403743315508, 'acc': 0.9762700534759359, 'cnnacc': 0.9762700534759359, 'obliacc': 0.7388034759358288}
Epoch 6/200
54000/54000 [==============================] - 72s 1ms/step - loss: 0.7499 - val_loss: 0.6350
evaluating ..
{'advacc': 0.7800802139037433, 'acc': 0.9732620320855615, 'cnnacc': 0.9732620320855615, 'obliacc': 0.7777406417112299}
Epoch 7/200
54000/54000 [==============================] - 72s 1ms/step - loss: 0.6651 - val_loss: 0.5762
evaluating ..
{'advacc': 0.7979612299465241, 'acc': 0.9742647058823529, 'cnnacc': 0.9742647058823529, 'obliacc': 0.7979612299465241}
Epoch 8/200
54000/54000 [==============================] - 73s 1ms/step - loss: 0.5972 - val_loss: 0.5054
evaluating ..
{'advacc': 0.8267045454545454, 'acc': 0.9784425133689839, 'cnnacc': 0.9784425133689839, 'obliacc': 0.8262032085561497}
Epoch 9/200
54000/54000 [==============================] - 72s 1ms/step - loss: 0.5378 - val_loss: 0.4529
evaluating ..
{'advacc': 0.8425802139037433, 'acc': 0.9764371657754011, 'cnnacc': 0.9764371657754011, 'obliacc': 0.8447526737967914}
Epoch 10/200
54000/54000 [==============================] - 72s 1ms/step - loss: 0.4879 - val_loss: 0.4083
evaluating ..
{'advacc': 0.8584558823529411, 'acc': 0.9781082887700535, 'cnnacc': 0.9781082887700535, 'obliacc': 0.859792780748663}
Epoch 11/200
54000/54000 [==============================] - 72s 1ms/step - loss: 0.4369 - val_loss: 0.3741
evaluating ..
{'advacc': 0.8734959893048129, 'acc': 0.9786096256684492, 'cnnacc': 0.9786096256684492, 'obliacc': 0.8733288770053476}
Epoch 12/200
54000/54000 [==============================] - 72s 1ms/step - loss: 0.3821 - val_loss: 0.3273
evaluating ..
{'advacc': 0.888201871657754, 'acc': 0.9817847593582888, 'cnnacc': 0.9817847593582888, 'obliacc': 0.8878676470588235}
Epoch 13/200
54000/54000 [==============================] - 72s 1ms/step - loss: 0.3332 - val_loss: 0.2863
evaluating ..
{'advacc': 0.9074197860962567, 'acc': 0.982620320855615, 'cnnacc': 0.982620320855615, 'obliacc': 0.9077540106951871}
Epoch 14/200
54000/54000 [==============================] - 72s 1ms/step - loss: 0.2933 - val_loss: 0.2574
evaluating ..
{'advacc': 0.9134358288770054, 'acc': 0.9827874331550802, 'cnnacc': 0.9827874331550802, 'obliacc': 0.914605614973262}
Epoch 15/200
54000/54000 [==============================] - 72s 1ms/step - loss: 0.2611 - val_loss: 0.2255
evaluating ..
{'advacc': 0.9269719251336899, 'acc': 0.9821189839572193, 'cnnacc': 0.9821189839572193, 'obliacc': 0.9273061497326203}
Epoch 16/200
54000/54000 [==============================] - 72s 1ms/step - loss: 0.2342 - val_loss: 0.2170
evaluating ..
{'advacc': 0.9296457219251337, 'acc': 0.9857954545454546, 'cnnacc': 0.9857954545454546, 'obliacc': 0.9294786096256684}
Epoch 17/200
54000/54000 [==============================] - 72s 1ms/step - loss: 0.2143 - val_loss: 0.1926
evaluating ..
{'advacc': 0.9373328877005348, 'acc': 0.9854612299465241, 'cnnacc': 0.9854612299465241, 'obliacc': 0.9371657754010695}
Epoch 18/200
54000/54000 [==============================] - 72s 1ms/step - loss: 0.1986 - val_loss: 0.1847
evaluating ..
{'advacc': 0.9410093582887701, 'acc': 0.9854612299465241, 'cnnacc': 0.9854612299465241, 'obliacc': 0.9396724598930482}
Epoch 19/200
54000/54000 [==============================] - 72s 1ms/step - loss: 0.1853 - val_loss: 0.1785
evaluating ..
{'advacc': 0.9405080213903744, 'acc': 0.9859625668449198, 'cnnacc': 0.9859625668449198, 'obliacc': 0.9421791443850267}
Epoch 20/200
54000/54000 [==============================] - 72s 1ms/step - loss: 0.1751 - val_loss: 0.1635
evaluating ..
{'advacc': 0.9455213903743316, 'acc': 0.9852941176470589, 'cnnacc': 0.9852941176470589, 'obliacc': 0.9478609625668449}
Epoch 21/200
54000/54000 [==============================] - 72s 1ms/step - loss: 0.1653 - val_loss: 0.1705
evaluating ..
{'advacc': 0.9403409090909091, 'acc': 0.9883021390374331, 'cnnacc': 0.9883021390374331, 'obliacc': 0.9413435828877005}
ResNet29, [0,1], lr=1e-3, CNN training. 25s/epoch
| epoch | val acc |
|---|---|
| 1 | 0.5877 |
| 2 | 0.6367 |
| 3 | 0.6883 |
| 4 | 0.7451 |
| 5 | 0.7561 |
| 19 | 0.8551 |
| 32 | 0.8725 |
| 36 | 0.8803 |
| 41 (5e-4) | 0.8926 |
| 47 (2.5e-4) | 0.8998 |
Training CNN .. Epoch 1/100 2019-11-11 00:43:36.578144: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0 2019-11-11 00:43:36.699025: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 352/352 [==============================] - 29s 82ms/step - loss: 1.9791 - acc: 0.4671 - val_loss: 1.6090 - val_acc: 0.5877 Epoch 2/100 352/352 [==============================] - 25s 70ms/step - loss: 1.5147 - acc: 0.6096 - val_loss: 1.3841 - val_acc: 0.6367 Epoch 3/100 352/352 [==============================] - 25s 70ms/step - loss: 1.3245 - acc: 0.6613 - val_loss: 1.2141 - val_acc: 0.6883 Epoch 4/100 352/352 [==============================] - 25s 71ms/step - loss: 1.1883 - acc: 0.7021 - val_loss: 1.0623 - val_acc: 0.7451 Epoch 5/100 352/352 [==============================] - 26s 73ms/step - loss: 1.0824 - acc: 0.7349 - val_loss: 1.0019 - val_acc: 0.7561 Epoch 6/100 352/352 [==============================] - 25s 71ms/step - loss: 1.0061 - acc: 0.7591 - val_loss: 0.9417 - val_acc: 0.7799 Epoch 7/100 352/352 [==============================] - 25s 70ms/step - loss: 0.9415 - acc: 0.7804 - val_loss: 0.8922 - val_acc: 0.7852 Epoch 8/100 352/352 [==============================] - 25s 70ms/step - loss: 0.8970 - acc: 0.7940 - val_loss: 0.8488 - val_acc: 0.7975 Epoch 9/100 352/352 [==============================] - 25s 70ms/step - loss: 0.8520 - acc: 0.8052 - val_loss: 0.8246 - val_acc: 0.8080 Epoch 10/100 352/352 [==============================] - 25s 70ms/step - loss: 0.8167 - acc: 0.8162 - val_loss: 0.7933 - val_acc: 0.8176 Epoch 11/100 [54/3080] 352/352 [==============================] - 25s 70ms/step - loss: 0.7874 - acc: 0.8250 - val_loss: 0.7685 - val_acc: 0.8164 Epoch 12/100 352/352 [==============================] - 25s 71ms/step - loss: 0.7636 - acc: 0.8298 - val_loss: 0.7399 - val_acc: 0.8289 Epoch 13/100 352/352 [==============================] - 25s 71ms/step - loss: 0.7465 - acc: 0.8356 - val_loss: 0.7551 - val_acc: 0.8211 Epoch 14/100 352/352 [==============================] - 25s 71ms/step - loss: 0.7190 - acc: 0.8431 - val_loss: 0.7266 - val_acc: 0.8311 Epoch 15/100 352/352 [==============================] - 25s 71ms/step - loss: 0.7064 - acc: 0.8469 - val_loss: 0.7193 - val_acc: 0.8340 Epoch 16/100 352/352 [==============================] - 25s 72ms/step - loss: 0.6898 - acc: 0.8519 - val_loss: 0.6852 - val_acc: 0.8406 Epoch 17/100 352/352 [==============================] - 25s 72ms/step - loss: 0.6673 - acc: 0.8593 - val_loss: 0.6858 - val_acc: 0.8459 Epoch 18/100 352/352 [==============================] - 26s 73ms/step - loss: 0.6627 - acc: 0.8596 - val_loss: 0.6842 - val_acc: 0.8479 Epoch 19/100 352/352 [==============================] - 25s 72ms/step - loss: 0.6456 - acc: 0.8633 - val_loss: 0.6532 - val_acc: 0.8551 Epoch 20/100 352/352 [==============================] - 25s 72ms/step - loss: 0.6322 - acc: 0.8686 - val_loss: 0.6663 - val_acc: 0.8488 Epoch 21/100 352/352 [==============================] - 25s 71ms/step - loss: 0.6209 - acc: 0.8705 - val_loss: 0.6652 - val_acc: 0.8492 Epoch 22/100 352/352 [==============================] - 25s 70ms/step - loss: 0.6160 - acc: 0.8716 - val_loss: 0.6691 - val_acc: 0.8490 Epoch 23/100 352/352 [==============================] - 25s 70ms/step - loss: 0.6057 - acc: 0.8743 - val_loss: 0.6428 - val_acc: 0.8598 Epoch 24/100 352/352 [==============================] - 25s 71ms/step - loss: 0.5901 - acc: 0.8808 - val_loss: 0.6492 - val_acc: 0.8559 Epoch 25/100 352/352 [==============================] - 25s 71ms/step - loss: 0.5860 - acc: 0.8814 - val_loss: 0.6317 - val_acc: 0.8545 Epoch 26/100 352/352 [==============================] - 25s 70ms/step - loss: 0.5842 - acc: 0.8805 - val_loss: 0.6253 - val_acc: 0.8531 Epoch 27/100 352/352 [==============================] - 25s 71ms/step - loss: 0.5673 - acc: 0.8866 - val_loss: 0.6214 - val_acc: 0.8652 Epoch 28/100 352/352 [==============================] - 25s 70ms/step - loss: 0.5644 - acc: 0.8873 - val_loss: 0.6152 - val_acc: 0.8607 Epoch 29/100 352/352 [==============================] - 25s 71ms/step - loss: 0.5594 - acc: 0.8890 - val_loss: 0.6426 - val_acc: 0.8607 Epoch 30/100 352/352 [==============================] - 25s 72ms/step - loss: 0.5530 - acc: 0.8911 - val_loss: 0.6093 - val_acc: 0.8709 Epoch 31/100 352/352 [==============================] - 25s 72ms/step - loss: 0.5413 - acc: 0.8938 - val_loss: 0.6106 - val_acc: 0.8668 Epoch 32/100 352/352 [==============================] - 25s 71ms/step - loss: 0.5361 - acc: 0.8944 - val_loss: 0.5932 - val_acc: 0.8725 Epoch 33/100 352/352 [==============================] - 25s 71ms/step - loss: 0.5295 - acc: 0.8979 - val_loss: 0.5969 - val_acc: 0.8744 Epoch 34/100 352/352 [==============================] - 25s 72ms/step - loss: 0.5260 - acc: 0.8991 - val_loss: 0.5853 - val_acc: 0.8766 Epoch 35/100 352/352 [==============================] - 25s 72ms/step - loss: 0.5262 - acc: 0.8986 - val_loss: 0.6231 - val_acc: 0.8676 Epoch 36/100 352/352 [==============================] - 25s 72ms/step - loss: 0.5185 - acc: 0.9011 - val_loss: 0.5846 - val_acc: 0.8803 Epoch 37/100 352/352 [==============================] - 25s 72ms/step - loss: 0.5168 - acc: 0.9012 - val_loss: 0.6062 - val_acc: 0.8684 Epoch 38/100 352/352 [==============================] - 25s 71ms/step - loss: 0.5078 - acc: 0.9046 - val_loss: 0.6204 - val_acc: 0.8605 Epoch 39/100 352/352 [==============================] - 25s 70ms/step - loss: 0.5082 - acc: 0.9052 - val_loss: 0.6036 - val_acc: 0.8631 Epoch 40/100 352/352 [==============================] - 25s 70ms/step - loss: 0.5048 - acc: 0.9046 - val_loss: 0.6020 - val_acc: 0.8705 Epoch 00040: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257. Epoch 41/100 352/352 [==============================] - 25s 71ms/step - loss: 0.4295 - acc: 0.9311 - val_loss: 0.5303 - val_acc: 0.8926 Epoch 42/100 352/352 [==============================] - 25s 70ms/step - loss: 0.4035 - acc: 0.9373 - val_loss: 0.5214 - val_acc: 0.8943 Epoch 43/100 352/352 [==============================] - 25s 70ms/step - loss: 0.3957 - acc: 0.9389 - val_loss: 0.5305 - val_acc: 0.8928 Epoch 44/100 352/352 [==============================] - 25s 70ms/step - loss: 0.3843 - acc: 0.9403 - val_loss: 0.5294 - val_acc: 0.8914 Epoch 45/100 352/352 [==============================] - 25s 71ms/step - loss: 0.3776 - acc: 0.9429 - val_loss: 0.5339 - val_acc: 0.8900 Epoch 46/100 352/352 [==============================] - 25s 70ms/step - loss: 0.3686 - acc: 0.9448 - val_loss: 0.5297 - val_acc: 0.8920 Epoch 00046: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628. Epoch 47/100 352/352 [==============================] - 25s 72ms/step - loss: 0.3301 - acc: 0.9556 - val_loss: 0.4967 - val_acc: 0.8998 Epoch 48/100 352/352 [==============================] - 25s 72ms/step - loss: 0.3143 - acc: 0.9617 - val_loss: 0.4995 - val_acc: 0.9008 Epoch 49/100 352/352 [==============================] - 25s 70ms/step - loss: 0.3061 - acc: 0.9628 - val_loss: 0.4930 - val_acc: 0.9016 Epoch 50/100 352/352 [==============================] - 25s 71ms/step - loss: 0.2991 - acc: 0.9644 - val_loss: 0.5061 - val_acc: 0.8959
Resnet29, [0,1], lr=2e-3, itadv, trained until early stop, 88 epochs. 160s/epoch. Note that 4e-4 is Madry’s starting lr.
| epoch | adv acc | clean acc |
|---|---|---|
| 1 (2e-3) | 0.2312 | 0.3425 |
| 2 | 0.2698 | 0.3758 |
| 10 | 0.3562 | 0.5401 |
| 16 | 0.3846 | 0.5739 |
| 23 | 0.3984 | 0.6170 |
| 27 (1e-3) | 0.4139 | 0.6336 |
| 33 | 0.4221 | 0.6430 |
| 37 (5e-4) | 0.4353 | 0.6755 |
| 43 | 0.4353 | 0.6863 |
| 47 (2.5e-4) | 0.4471 | 0.7003 |
| 60 | 0.4511 | 0.7047 |
| 71 (6.25e-5) | 0.4573 | 0.7167 |
| 76 (5e-5) | 0.4573 | 0.7133 |
| 81 (best) | 0.4593 | 0.7200 |
| 88 | 0.4565 | 0.7236 |
- epoch 1: ‘advacc’: 0.2311698717948718, ‘acc’: 0.3425480769230769
- epoch 2: ‘advacc’: 0.2698317307692308, ‘acc’: 0.37580128205128205
- epoch 10: ‘advacc’: 0.3561698717948718, ‘acc’: 0.5400641025641025
- epoch 16: ‘advacc’: 0.38461538461538464, ‘acc’: 0.5739182692307693
- epoch 23 (highest before lr reduce): ‘advacc’: 0.3984375, ‘acc’: 0.6169871794871795
- epoch 27 (lr reduced to 1e-3): ‘advacc’: 0.41386217948717946, ‘acc’: 0.633613782051282
- epoch 33: ‘advacc’: 0.42207532051282054, ‘acc’: 0.6430288461538461
- epoch 37 (lr reduced to 5e-4): ‘advacc’: 0.43529647435897434, ‘acc’: 0.6754807692307693
- epoch 43: ‘advacc’: 0.43529647435897434, ‘acc’: 0.6862980769230769
- epoch 47 (lr reduced to 2.5e-4): ‘advacc’: 0.44711538461538464, ‘acc’: 0.7003205128205128
- epoch 60: ‘advacc’: 0.4511217948717949, ‘acc’: 0.7047275641025641
- epoch 64 (lr reduced to 1.25e-4): ‘advacc’: 0.45592948717948717, ‘acc’: 0.7107371794871795
- epoch 71 (lr reduced to 0.625e-4): ‘advacc’: 0.4573317307692308, ‘acc’: 0.7167467948717948
- epoch 76 (lr reduced to 5e-5): ‘advacc’: 0.4573317307692308, ‘acc’: 0.7133413461538461
- epoch 81 (best): ‘advacc’: 0.4593349358974359, ‘acc’: 0.7199519230769231
- epoch 88: ‘advacc’: 0.45653044871794873, ‘acc’: 0.7235576923076923
Trainng AdvAE saved_models/CIFAR10-resnet29-identityAE-ItAdv-AdvAE.hdf5 ..
!!!!!!! Training ItAdv models, setting CNN trainable
!!!!! Training on datagen with data augmentation
Epoch 1/200
2019-11-11 01:48:52.426497: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2019-11-11 01:48:52.544670: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
352/352 [==============================] - 167s 474ms/step - loss: 2.4792 - val_loss: 2.1857
evaluating ..
{'advacc': 0.2311698717948718, 'acc': 0.3425480769230769, 'cnnacc': 0.3425480769230769, 'obliacc': 0.2309695512820513}
Epoch 2/200
352/352 [==============================] - 160s 455ms/step - loss: 2.1418 - val_loss: 2.0547
evaluating ..
{'advacc': 0.2698317307692308, 'acc': 0.37580128205128205, 'cnnacc': 0.37580128205128205, 'obliacc': 0.26963141025641024}
Epoch 3/200
352/352 [==============================] - 160s 455ms/step - loss: 2.0596 - val_loss: 1.9764
evaluating ..
{'advacc': 0.281650641025641, 'acc': 0.41806891025641024, 'cnnacc': 0.41806891025641024, 'obliacc': 0.2824519230769231}
Epoch 4/200
352/352 [==============================] - 160s 455ms/step - loss: 2.0052 - val_loss: 1.9268
evaluating ..
{'advacc': 0.2994791666666667, 'acc': 0.4427083333333333, 'cnnacc': 0.4427083333333333, 'obliacc': 0.2994791666666667}
Epoch 5/200
352/352 [==============================] - 160s 455ms/step - loss: 1.9722 - val_loss: 1.9073
evaluating ..
{'advacc': 0.30128205128205127, 'acc': 0.45532852564102566, 'cnnacc': 0.45532852564102566, 'obliacc': 0.3016826923076923}
Epoch 6/200
352/352 [==============================] - 160s 455ms/step - loss: 1.9412 - val_loss: 1.8753
evaluating ..
{'advacc': 0.31430288461538464, 'acc': 0.4833733974358974, 'cnnacc': 0.4833733974358974, 'obliacc': 0.3145032051282051}
Epoch 7/200
352/352 [==============================] - 160s 455ms/step - loss: 1.9166 - val_loss: 1.8459
evaluating ..
{'advacc': 0.328525641025641, 'acc': 0.4905849358974359, 'cnnacc': 0.4905849358974359, 'obliacc': 0.3297275641025641}
Epoch 8/200
352/352 [==============================] - 160s 456ms/step - loss: 1.8991 - val_loss: 1.8128
evaluating ..
{'advacc': 0.33673878205128205, 'acc': 0.5032051282051282, 'cnnacc': 0.5032051282051282, 'obliacc': 0.3359375}
Epoch 9/200
352/352 [==============================] - 160s 455ms/step - loss: 1.8800 - val_loss: 1.8097
evaluating ..
{'advacc': 0.3423477564102564, 'acc': 0.5152243589743589, 'cnnacc': 0.5152243589743589, 'obliacc': 0.3425480769230769}
Epoch 10/200
352/352 [==============================] - 160s 456ms/step - loss: 1.8632 - val_loss: 1.7802
evaluating ..
{'advacc': 0.3561698717948718, 'acc': 0.5400641025641025, 'cnnacc': 0.5400641025641025, 'obliacc': 0.35496794871794873}
Epoch 11/200
352/352 [==============================] - 160s 455ms/step - loss: 1.8489 - val_loss: 1.7714
evaluating ..
{'advacc': 0.35376602564102566, 'acc': 0.5332532051282052, 'cnnacc': 0.5332532051282052, 'obliacc': 0.35396634615384615}
Epoch 12/200
352/352 [==============================] - 160s 456ms/step - loss: 1.8409 - val_loss: 1.7562
evaluating ..
{'advacc': 0.36217948717948717, 'acc': 0.5436698717948718, 'cnnacc': 0.5436698717948718, 'obliacc': 0.3623798076923077}
Epoch 13/200
352/352 [==============================] - 160s 455ms/step - loss: 1.8282 - val_loss: 1.7386
evaluating ..
{'advacc': 0.36378205128205127, 'acc': 0.5590945512820513, 'cnnacc': 0.5590945512820513, 'obliacc': 0.36498397435897434}
Epoch 14/200
352/352 [==============================] - 160s 455ms/step - loss: 1.8187 - val_loss: 1.7194
evaluating ..
{'advacc': 0.37319711538461536, 'acc': 0.5847355769230769, 'cnnacc': 0.5847355769230769, 'obliacc': 0.37279647435897434}
Epoch 15/200
352/352 [==============================] - 160s 455ms/step - loss: 1.8077 - val_loss: 1.7249
evaluating ..
{'advacc': 0.36498397435897434, 'acc': 0.5707131410256411, 'cnnacc': 0.5707131410256411, 'obliacc': 0.36538461538461536}
Epoch 16/200
352/352 [==============================] - 160s 455ms/step - loss: 1.8018 - val_loss: 1.7117
evaluating ..
{'advacc': 0.38461538461538464, 'acc': 0.5739182692307693, 'cnnacc': 0.5739182692307693, 'obliacc': 0.38501602564102566}
Epoch 17/200
352/352 [==============================] - 160s 455ms/step - loss: 1.7941 - val_loss: 1.7063
evaluating ..
{'advacc': 0.3764022435897436, 'acc': 0.563301282051282, 'cnnacc': 0.563301282051282, 'obliacc': 0.37720352564102566}
Epoch 18/200
352/352 [==============================] - 160s 455ms/step - loss: 1.7854 - val_loss: 1.7065
evaluating ..
{'advacc': 0.3717948717948718, 'acc': 0.5779246794871795, 'cnnacc': 0.5779246794871795, 'obliacc': 0.3703926282051282}
Epoch 19/200
352/352 [==============================] - 160s 455ms/step - loss: 1.7795 - val_loss: 1.7049
evaluating ..
{'advacc': 0.3766025641025641, 'acc': 0.5873397435897436, 'cnnacc': 0.5873397435897436, 'obliacc': 0.375400641025641}
Epoch 20/200
352/352 [==============================] - 160s 455ms/step - loss: 1.7737 - val_loss: 1.6928
evaluating ..
{'advacc': 0.38000801282051283, 'acc': 0.5933493589743589, 'cnnacc': 0.5933493589743589, 'obliacc': 0.3812099358974359}
Epoch 21/200
352/352 [==============================] - 160s 455ms/step - loss: 1.7669 - val_loss: 1.6682
evaluating ..
{'advacc': 0.3958333333333333, 'acc': 0.594551282051282, 'cnnacc': 0.594551282051282, 'obliacc': 0.39763621794871795}
Epoch 22/200
352/352 [==============================] - 160s 455ms/step - loss: 1.7634 - val_loss: 1.6811
evaluating ..
{'advacc': 0.38782051282051283, 'acc': 0.6049679487179487, 'cnnacc': 0.6049679487179487, 'obliacc': 0.3858173076923077}
Epoch 23/200
352/352 [==============================] - 160s 456ms/step - loss: 1.7567 - val_loss: 1.6620
evaluating ..
{'advacc': 0.3984375, 'acc': 0.6169871794871795, 'cnnacc': 0.6169871794871795, 'obliacc': 0.3974358974358974}
Epoch 24/200
352/352 [==============================] - 160s 456ms/step - loss: 1.7543 - val_loss: 1.6727
evaluating ..
{'advacc': 0.3922275641025641, 'acc': 0.6125801282051282, 'cnnacc': 0.6125801282051282, 'obliacc': 0.3920272435897436}
Epoch 25/200
352/352 [==============================] - 160s 455ms/step - loss: 1.7486 - val_loss: 1.6716
evaluating ..
{'advacc': 0.3932291666666667, 'acc': 0.602363782051282, 'cnnacc': 0.602363782051282, 'obliacc': 0.39443108974358976}
Epoch 26/200
352/352 [==============================] - 160s 455ms/step - loss: 1.7432 - val_loss: 1.6728
Epoch 00026: ReduceLROnPlateau reducing learning rate to 0.0010000000474974513.
evaluating ..
{'advacc': 0.39002403846153844, 'acc': 0.6177884615384616, 'cnnacc': 0.6177884615384616, 'obliacc': 0.39002403846153844}
Epoch 27/200
352/352 [==============================] - 160s 455ms/step - loss: 1.6911 - val_loss: 1.5999
evaluating ..
{'advacc': 0.41386217948717946, 'acc': 0.633613782051282, 'cnnacc': 0.633613782051282, 'obliacc': 0.41326121794871795}
Epoch 28/200
352/352 [==============================] - 160s 455ms/step - loss: 1.6782 - val_loss: 1.5953
evaluating ..
{'advacc': 0.4072516025641026, 'acc': 0.6338141025641025, 'cnnacc': 0.6338141025641025, 'obliacc': 0.40685096153846156}
Epoch 29/200
352/352 [==============================] - 160s 455ms/step - loss: 1.6727 - val_loss: 1.5948
evaluating ..
{'advacc': 0.4108573717948718, 'acc': 0.6358173076923077, 'cnnacc': 0.6358173076923077, 'obliacc': 0.4108573717948718}
Epoch 30/200
352/352 [==============================] - 160s 455ms/step - loss: 1.6656 - val_loss: 1.5914
evaluating ..
{'advacc': 0.4130608974358974, 'acc': 0.6290064102564102, 'cnnacc': 0.6290064102564102, 'obliacc': 0.41326121794871795}
Epoch 31/200
352/352 [==============================] - 160s 455ms/step - loss: 1.6580 - val_loss: 1.5843
evaluating ..
{'advacc': 0.41646634615384615, 'acc': 0.632011217948718, 'cnnacc': 0.632011217948718, 'obliacc': 0.4160657051282051}
Epoch 32/200
352/352 [==============================] - 160s 455ms/step - loss: 1.6537 - val_loss: 1.5693
evaluating ..
{'advacc': 0.4206730769230769, 'acc': 0.6582532051282052, 'cnnacc': 0.6582532051282052, 'obliacc': 0.4202724358974359}
Epoch 33/200
352/352 [==============================] - 160s 456ms/step - loss: 1.6499 - val_loss: 1.5681
evaluating ..
{'advacc': 0.42207532051282054, 'acc': 0.6430288461538461, 'cnnacc': 0.6430288461538461, 'obliacc': 0.4208733974358974}
Epoch 34/200
352/352 [==============================] - 160s 455ms/step - loss: 1.6436 - val_loss: 1.5684
evaluating ..
{'advacc': 0.4198717948717949, 'acc': 0.6588541666666666, 'cnnacc': 0.6588541666666666, 'obliacc': 0.41846955128205127}
Epoch 35/200
352/352 [==============================] - 160s 455ms/step - loss: 1.6425 - val_loss: 1.5686
evaluating ..
{'advacc': 0.42207532051282054, 'acc': 0.6522435897435898, 'cnnacc': 0.6522435897435898, 'obliacc': 0.4228766025641026}
Epoch 36/200
352/352 [==============================] - 160s 455ms/step - loss: 1.6425 - val_loss: 1.5694
Epoch 00036: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257.
evaluating ..
{'advacc': 0.4192708333333333, 'acc': 0.6464342948717948, 'cnnacc': 0.6464342948717948, 'obliacc': 0.4202724358974359}
Epoch 37/200
352/352 [==============================] - 160s 455ms/step - loss: 1.5979 - val_loss: 1.5257
evaluating ..
{'advacc': 0.43529647435897434, 'acc': 0.6754807692307693, 'cnnacc': 0.6754807692307693, 'obliacc': 0.4364983974358974}
Epoch 38/200
352/352 [==============================] - 160s 455ms/step - loss: 1.5886 - val_loss: 1.5238
evaluating ..
{'advacc': 0.4332932692307692, 'acc': 0.6744791666666666, 'cnnacc': 0.6744791666666666, 'obliacc': 0.43369391025641024}
Epoch 39/200
352/352 [==============================] - 160s 456ms/step - loss: 1.5865 - val_loss: 1.5165
evaluating ..
{'advacc': 0.43569711538461536, 'acc': 0.6728766025641025, 'cnnacc': 0.6728766025641025, 'obliacc': 0.4358974358974359}
Epoch 40/200
352/352 [==============================] - 160s 455ms/step - loss: 1.5780 - val_loss: 1.5134
evaluating ..
{'advacc': 0.4310897435897436, 'acc': 0.6844951923076923, 'cnnacc': 0.6844951923076923, 'obliacc': 0.43149038461538464}
Epoch 41/200
352/352 [==============================] - 160s 456ms/step - loss: 1.5773 - val_loss: 1.5127
evaluating ..
{'advacc': 0.43309294871794873, 'acc': 0.6814903846153846, 'cnnacc': 0.6814903846153846, 'obliacc': 0.43309294871794873}
Epoch 42/200
352/352 [==============================] - 160s 455ms/step - loss: 1.5703 - val_loss: 1.5075
evaluating ..
{'advacc': 0.4385016025641026, 'acc': 0.6816907051282052, 'cnnacc': 0.6816907051282052, 'obliacc': 0.4389022435897436}
Epoch 43/200
352/352 [==============================] - 160s 455ms/step - loss: 1.5694 - val_loss: 1.5016
evaluating ..
{'advacc': 0.43529647435897434, 'acc': 0.6862980769230769, 'cnnacc': 0.6862980769230769, 'obliacc': 0.4358974358974359}
Epoch 44/200
352/352 [==============================] - 160s 455ms/step - loss: 1.5645 - val_loss: 1.5089
evaluating ..
{'advacc': 0.43369391025641024, 'acc': 0.6820913461538461, 'cnnacc': 0.6820913461538461, 'obliacc': 0.43469551282051283}
Epoch 45/200
352/352 [==============================] - 160s 455ms/step - loss: 1.5667 - val_loss: 1.5045
evaluating ..
{'advacc': 0.43349358974358976, 'acc': 0.6905048076923077, 'cnnacc': 0.6905048076923077, 'obliacc': 0.4326923076923077}
Epoch 46/200
352/352 [==============================] - 160s 455ms/step - loss: 1.5611 - val_loss: 1.5047
Epoch 00046: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628.
evaluating ..
{'advacc': 0.4316907051282051, 'acc': 0.6754807692307693, 'cnnacc': 0.6754807692307693, 'obliacc': 0.43209134615384615}
Epoch 47/200
352/352 [==============================] - 160s 455ms/step - loss: 1.5294 - val_loss: 1.4807
evaluating ..
{'advacc': 0.44711538461538464, 'acc': 0.7003205128205128, 'cnnacc': 0.7003205128205128, 'obliacc': 0.4495192307692308}
Epoch 48/200
352/352 [==============================] - 160s 455ms/step - loss: 1.5251 - val_loss: 1.4783
evaluating ..
{'advacc': 0.4433092948717949, 'acc': 0.6979166666666666, 'cnnacc': 0.6979166666666666, 'obliacc': 0.44350961538461536}
Epoch 49/200
352/352 [==============================] - 160s 455ms/step - loss: 1.5202 - val_loss: 1.4778
evaluating ..
{'advacc': 0.44591346153846156, 'acc': 0.6973157051282052, 'cnnacc': 0.6973157051282052, 'obliacc': 0.44611378205128205}
Epoch 50/200
352/352 [==============================] - 160s 455ms/step - loss: 1.5171 - val_loss: 1.4744
evaluating ..
{'advacc': 0.44751602564102566, 'acc': 0.7049278846153846, 'cnnacc': 0.7049278846153846, 'obliacc': 0.4469150641025641}
Epoch 51/200
352/352 [==============================] - 160s 455ms/step - loss: 1.5137 - val_loss: 1.4716
evaluating ..
{'advacc': 0.44471153846153844, 'acc': 0.7025240384615384, 'cnnacc': 0.7025240384615384, 'obliacc': 0.44290865384615385}
Epoch 52/200
352/352 [==============================] - 160s 456ms/step - loss: 1.5126 - val_loss: 1.4749
evaluating ..
{'advacc': 0.4495192307692308, 'acc': 0.6989182692307693, 'cnnacc': 0.6989182692307693, 'obliacc': 0.4485176282051282}
Epoch 53/200
352/352 [==============================] - 160s 455ms/step - loss: 1.5089 - val_loss: 1.4710
evaluating ..
{'advacc': 0.44310897435897434, 'acc': 0.7073317307692307, 'cnnacc': 0.7073317307692307, 'obliacc': 0.4423076923076923}
Epoch 54/200
352/352 [==============================] - 160s 455ms/step - loss: 1.5066 - val_loss: 1.4675
evaluating ..
{'advacc': 0.4453125, 'acc': 0.7015224358974359, 'cnnacc': 0.7015224358974359, 'obliacc': 0.4467147435897436}
Epoch 55/200
352/352 [==============================] - 160s 456ms/step - loss: 1.5028 - val_loss: 1.4705
evaluating ..
{'advacc': 0.4439102564102564, 'acc': 0.702323717948718, 'cnnacc': 0.702323717948718, 'obliacc': 0.44511217948717946}
Epoch 56/200
352/352 [==============================] - 160s 455ms/step - loss: 1.5008 - val_loss: 1.4691
evaluating ..
{'advacc': 0.4501201923076923, 'acc': 0.703125, 'cnnacc': 0.703125, 'obliacc': 0.45072115384615385}
Epoch 57/200
352/352 [==============================] - 160s 455ms/step - loss: 1.4996 - val_loss: 1.4653
evaluating ..
{'advacc': 0.4495192307692308, 'acc': 0.7105368589743589, 'cnnacc': 0.7105368589743589, 'obliacc': 0.44971955128205127}
Epoch 58/200
352/352 [==============================] - 160s 455ms/step - loss: 1.4966 - val_loss: 1.4651
evaluating ..
{'advacc': 0.44751602564102566, 'acc': 0.7047275641025641, 'cnnacc': 0.7047275641025641, 'obliacc': 0.4467147435897436}
Epoch 59/200
352/352 [==============================] - 160s 455ms/step - loss: 1.4971 - val_loss: 1.4684
evaluating ..
{'advacc': 0.4469150641025641, 'acc': 0.6993189102564102, 'cnnacc': 0.6993189102564102, 'obliacc': 0.4485176282051282}
Epoch 60/200
352/352 [==============================] - 160s 455ms/step - loss: 1.4932 - val_loss: 1.4588
evaluating ..
{'advacc': 0.4511217948717949, 'acc': 0.7047275641025641, 'cnnacc': 0.7047275641025641, 'obliacc': 0.45132211538461536}
Epoch 61/200
352/352 [==============================] - 160s 455ms/step - loss: 1.4943 - val_loss: 1.4664
evaluating ..
{'advacc': 0.44471153846153844, 'acc': 0.7079326923076923, 'cnnacc': 0.7079326923076923, 'obliacc': 0.4441105769230769}
Epoch 62/200
352/352 [==============================] - 160s 456ms/step - loss: 1.4919 - val_loss: 1.4625
evaluating ..
{'advacc': 0.4485176282051282, 'acc': 0.7063301282051282, 'cnnacc': 0.7063301282051282, 'obliacc': 0.4485176282051282}
Epoch 63/200
352/352 [==============================] - 160s 455ms/step - loss: 1.4902 - val_loss: 1.4591
Epoch 00063: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814.
evaluating ..
{'advacc': 0.44811698717948717, 'acc': 0.7089342948717948, 'cnnacc': 0.7089342948717948, 'obliacc': 0.44711538461538464}
Epoch 64/200
352/352 [==============================] - 160s 455ms/step - loss: 1.4716 - val_loss: 1.4486
evaluating ..
{'advacc': 0.45592948717948717, 'acc': 0.7107371794871795, 'cnnacc': 0.7107371794871795, 'obliacc': 0.45532852564102566}
Epoch 65/200
352/352 [==============================] - 160s 455ms/step - loss: 1.4687 - val_loss: 1.4514
evaluating ..
{'advacc': 0.452724358974359, 'acc': 0.7153445512820513, 'cnnacc': 0.7153445512820513, 'obliacc': 0.4547275641025641}
Epoch 66/200
352/352 [==============================] - 160s 455ms/step - loss: 1.4674 - val_loss: 1.4488
evaluating ..
{'advacc': 0.45252403846153844, 'acc': 0.7099358974358975, 'cnnacc': 0.7099358974358975, 'obliacc': 0.45232371794871795}
Epoch 67/200
352/352 [==============================] - 160s 456ms/step - loss: 1.4656 - val_loss: 1.4441
evaluating ..
{'advacc': 0.45292467948717946, 'acc': 0.7137419871794872, 'cnnacc': 0.7137419871794872, 'obliacc': 0.4545272435897436}
Epoch 68/200
352/352 [==============================] - 160s 455ms/step - loss: 1.4634 - val_loss: 1.4468
evaluating ..
{'advacc': 0.4479166666666667, 'acc': 0.7147435897435898, 'cnnacc': 0.7147435897435898, 'obliacc': 0.4495192307692308}
Epoch 69/200
352/352 [==============================] - 160s 455ms/step - loss: 1.4612 - val_loss: 1.4488
evaluating ..
{'advacc': 0.4499198717948718, 'acc': 0.7173477564102564, 'cnnacc': 0.7173477564102564, 'obliacc': 0.45092147435897434}
Epoch 70/200
352/352 [==============================] - 160s 455ms/step - loss: 1.4601 - val_loss: 1.4441
Epoch 00070: ReduceLROnPlateau reducing learning rate to 6.25000029685907e-05.
evaluating ..
{'advacc': 0.4511217948717949, 'acc': 0.7141426282051282, 'cnnacc': 0.7141426282051282, 'obliacc': 0.45092147435897434}
Epoch 71/200
352/352 [==============================] - 160s 455ms/step - loss: 1.4480 - val_loss: 1.4395
evaluating ..
{'advacc': 0.4573317307692308, 'acc': 0.7167467948717948, 'cnnacc': 0.7167467948717948, 'obliacc': 0.4567307692307692}
Epoch 72/200
352/352 [==============================] - 160s 456ms/step - loss: 1.4473 - val_loss: 1.4378
evaluating ..
{'advacc': 0.4545272435897436, 'acc': 0.7141426282051282, 'cnnacc': 0.7141426282051282, 'obliacc': 0.4557291666666667}
Epoch 73/200
352/352 [==============================] - 160s 455ms/step - loss: 1.4445 - val_loss: 1.4405
evaluating ..
{'advacc': 0.4563301282051282, 'acc': 0.7153445512820513, 'cnnacc': 0.7153445512820513, 'obliacc': 0.4563301282051282}
Epoch 74/200
352/352 [==============================] - 160s 455ms/step - loss: 1.4431 - val_loss: 1.4394
evaluating ..
{'advacc': 0.45853365384615385, 'acc': 0.7147435897435898, 'cnnacc': 0.7147435897435898, 'obliacc': 0.4597355769230769}
Epoch 75/200
352/352 [==============================] - 160s 455ms/step - loss: 1.4428 - val_loss: 1.4381
Epoch 00075: ReduceLROnPlateau reducing learning rate to 5e-05.
evaluating ..
{'advacc': 0.4567307692307692, 'acc': 0.7155448717948718, 'cnnacc': 0.7155448717948718, 'obliacc': 0.4567307692307692}
Epoch 76/200
352/352 [==============================] - 160s 455ms/step - loss: 1.4391 - val_loss: 1.4367
evaluating ..
{'advacc': 0.4573317307692308, 'acc': 0.7133413461538461, 'cnnacc': 0.7133413461538461, 'obliacc': 0.45713141025641024}
Epoch 77/200
352/352 [==============================] - 160s 455ms/step - loss: 1.4383 - val_loss: 1.4352
evaluating ..
{'advacc': 0.4561298076923077, 'acc': 0.7207532051282052, 'cnnacc': 0.7207532051282052, 'obliacc': 0.45653044871794873}
Epoch 78/200
352/352 [==============================] - 160s 455ms/step - loss: 1.4392 - val_loss: 1.4385
evaluating ..
{'advacc': 0.45592948717948717, 'acc': 0.7211538461538461, 'cnnacc': 0.7211538461538461, 'obliacc': 0.4557291666666667}
Epoch 79/200
352/352 [==============================] - 160s 455ms/step - loss: 1.4370 - val_loss: 1.4351
evaluating ..
{'advacc': 0.4577323717948718, 'acc': 0.7191506410256411, 'cnnacc': 0.7191506410256411, 'obliacc': 0.4567307692307692}
Epoch 80/200
352/352 [==============================] - 160s 455ms/step - loss: 1.4380 - val_loss: 1.4363
evaluating ..
{'advacc': 0.45853365384615385, 'acc': 0.7157451923076923, 'cnnacc': 0.7157451923076923, 'obliacc': 0.45753205128205127}
Epoch 81/200
352/352 [==============================] - 160s 455ms/step - loss: 1.4354 - val_loss: 1.4332
evaluating ..
{'advacc': 0.4593349358974359, 'acc': 0.7199519230769231, 'cnnacc': 0.7199519230769231, 'obliacc': 0.45813301282051283}
Epoch 82/200
352/352 [==============================] - 160s 455ms/step - loss: 1.4352 - val_loss: 1.4354
evaluating ..
{'advacc': 0.45592948717948717, 'acc': 0.7177483974358975, 'cnnacc': 0.7177483974358975, 'obliacc': 0.45693108974358976}
Epoch 83/200
352/352 [==============================] - 160s 455ms/step - loss: 1.4356 - val_loss: 1.4343
evaluating ..
{'advacc': 0.4573317307692308, 'acc': 0.7223557692307693, 'cnnacc': 0.7223557692307693, 'obliacc': 0.45693108974358976}
Epoch 84/200
352/352 [==============================] - 160s 455ms/step - loss: 1.4362 - val_loss: 1.4338
evaluating ..
{'advacc': 0.4557291666666667, 'acc': 0.7207532051282052, 'cnnacc': 0.7207532051282052, 'obliacc': 0.45753205128205127}
Epoch 85/200
352/352 [==============================] - 160s 456ms/step - loss: 1.4330 - val_loss: 1.4355
evaluating ..
{'advacc': 0.4573317307692308, 'acc': 0.7169471153846154, 'cnnacc': 0.7169471153846154, 'obliacc': 0.4573317307692308}
Epoch 86/200
352/352 [==============================] - 160s 455ms/step - loss: 1.4306 - val_loss: 1.4347
evaluating ..
{'advacc': 0.45392628205128205, 'acc': 0.7167467948717948, 'cnnacc': 0.7167467948717948, 'obliacc': 0.4541266025641026}
Epoch 87/200
352/352 [==============================] - 160s 456ms/step - loss: 1.4320 - val_loss: 1.4356
evaluating ..
{'advacc': 0.45532852564102566, 'acc': 0.7207532051282052, 'cnnacc': 0.7207532051282052, 'obliacc': 0.45492788461538464}
Epoch 88/200
352/352 [==============================] - 160s 455ms/step - loss: 1.4326 - val_loss: 1.4356
evaluating ..
{'advacc': 0.45653044871794873, 'acc': 0.7235576923076923, 'cnnacc': 0.7235576923076923, 'obliacc': 0.45753205128205127}
Restoring model weights from the end of the best epoch
Epoch 00088: early stopping
Saving model to saved_models/CIFAR10-resnet29-CNN.hdf5-identityAE-ItAdvCNN.hdf5 ..
saving to saved_models/CIFAR10-resnet29-identityAE-ItAdv-AdvAE.hdf5 ..
Resnet29, [0,1], lr=1e-4, itadv. The lr is not a good one.
For lr=0.1 in [0,255], it is 0.1/255=4e-4
I probably try 2e-3, 4e-4
Epoch 1/200
2019-11-11 01:07:19.485215: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2019-11-11 01:07:19.601530: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
352/352 [==============================] - 170s 482ms/step - loss: 3.0833 - val_loss: 2.8146
evaluating ..
{'advacc': 0.14743589743589744, 'acc': 0.2592147435897436, 'cnnacc': 0.2592147435897436, 'obliacc': 0.14703525641025642}
Epoch 2/200
352/352 [==============================] - 163s 464ms/step - loss: 2.7599 - val_loss: 2.7004
evaluating ..
{'advacc': 0.1872996794871795, 'acc': 0.28345352564102566, 'cnnacc': 0.28345352564102566, 'obliacc': 0.1875}
Epoch 3/200
352/352 [==============================] - 162s 462ms/step - loss: 2.6675 - val_loss: 2.6132
evaluating ..
{'advacc': 0.2033253205128205, 'acc': 0.31490384615384615, 'cnnacc': 0.31490384615384615, 'obliacc': 0.2029246794871795}
Epoch 4/200
352/352 [==============================] - 163s 464ms/step - loss: 2.5882 - val_loss: 2.5300
evaluating ..
{'advacc': 0.22576121794871795, 'acc': 0.3233173076923077, 'cnnacc': 0.3233173076923077, 'obliacc': 0.22536057692307693}
Epoch 5/200
352/352 [==============================] - 163s 462ms/step - loss: 2.5146 - val_loss: 2.4562
evaluating ..
{'advacc': 0.24358974358974358, 'acc': 0.36438301282051283, 'cnnacc': 0.36438301282051283, 'obliacc': 0.24358974358974358}
Epoch 6/200
352/352 [==============================] - 164s 465ms/step - loss: 2.4527 - val_loss: 2.3949
evaluating ..
{'advacc': 0.2532051282051282, 'acc': 0.38040865384615385, 'cnnacc': 0.38040865384615385, 'obliacc': 0.25220352564102566}
Epoch 7/200
352/352 [==============================] - 162s 461ms/step - loss: 2.4032 - val_loss: 2.3408
evaluating ..
{'advacc': 0.2598157051282051, 'acc': 0.38822115384615385, 'cnnacc': 0.38822115384615385, 'obliacc': 0.26061698717948717}
Epoch 8/200
352/352 [==============================] - 164s 465ms/step - loss: 2.3577 - val_loss: 2.3035
evaluating ..
{'advacc': 0.2718349358974359, 'acc': 0.41386217948717946, 'cnnacc': 0.41386217948717946, 'obliacc': 0.27283653846153844}
Epoch 9/200
352/352 [==============================] - 176s 500ms/step - loss: 2.3215 - val_loss: 2.2662
evaluating ..
{'advacc': 0.27123397435897434, 'acc': 0.42367788461538464, 'cnnacc': 0.42367788461538464, 'obliacc': 0.27163461538461536}
Epoch 10/200
352/352 [==============================] - 164s 466ms/step - loss: 2.2896 - val_loss: 2.2346
evaluating ..
{'advacc': 0.2796474358974359, 'acc': 0.42948717948717946, 'cnnacc': 0.42948717948717946, 'obliacc': 0.2800480769230769}
Epoch 11/200
352/352 [==============================] - 166s 471ms/step - loss: 2.2618 - val_loss: 2.2081
evaluating ..
{'advacc': 0.28685897435897434, 'acc': 0.4387019230769231, 'cnnacc': 0.4387019230769231, 'obliacc': 0.2876602564102564}