!kur --version
Kur, by Deepgram -- deep learning made easy Version: 0.3.0 Homepage: https://kur.deepgram.com
%%writefile dlnd_p2.yml
---
settings:
# Where to get the data
cifar: &cifar
url: "https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz"
checksum: "6d958be074577803d12ecdefd02955f39262c83c16fe9348329d7fe0b5c001ce"
path: "~/kur"
# I am tempting to read supplier source code to understand better how to use
# each supplier (I see a few available, csv, pickle), do you think
# it is a good idea to spend time on? #################################################
# Backend to use
backend:
name: keras # how many backend does kur have, and what different do they make? #####################
# some complaint about keras performance, do you think it is an issue? #################
# Hyperparameters
cnn:
kernels: 20
size: [5, 5]
strides: [2, 2]
pool:
size: [2,2]
strides: [2,2]
type: "max"
model:
- input: images
- convolution:
kernels: "{{cnn.kernels}}"
size: "{{cnn.size}}"
strides: "{{ cnn.strides }}"
- activation: relu
- pool:
size: "{{pool.size}}"
strides: "{{pool.strides}}"
type: "{{pool.type}}"
- flatten:
- dense: 15
- dense: 10
- activation: softmax
name: labels
train:
data:
- cifar:
<<: *cifar
parts: [1] # only use dataset part 1 to train
provider:
batch_size: 128
num_batches: 1000 # the entire part 1 will be used
log: t3/cifar-log
epochs: 20
weights:
initial: t3/cifar.best.valid.w # do you think is it essential to implement weight visualization? #############
best: t3/cifar.best.train.w # visualizing weights on different layers is used in explaining cnn on image###
last: t3/cifar.last.w # or is it not important for help experimenting on models #####################
optimizer:
name: adam
learning_rate: 0.001
validate:
data:
- cifar:
<<: *cifar
parts: 5 # only use dataset part 5 as validation set
provider:
num_batches: 50 # the project 2 only used 5000 data points as validation set
weights: t3/cifar.best.valid.w
test: &test
data:
- cifar:
<<: *cifar
parts: test
weights: t3/cifar.best.valid.w
provider:
num_batches: 1000 # the entire part test will be used
evaluate:
<<: *test
destination: t3/cifar.results.pkl
loss:
- target: labels # in the project: training loss and valid_accuracy are printed #############
name: categorical_crossentropy # this should be a matter of personal taste, won't really affect anything##
...
Overwriting dlnd_p2.yml
%pwd
'/Users/Natsume/Downloads/kur_road'
%ls
Kur_Road.ipynb dlnd_p2_dropout.yml t1/ cifar-log/ dlnd_p2_kur.ipynb t2/ dlnd_p2.yml kur/ t3/
!kur -v train dlnd_p2.yml
[INFO 2017-02-28 22:28:48,972 kur.kurfile:638] Parsing source: dlnd_p2.yml, included by top-level. [INFO 2017-02-28 22:28:48,989 kur.kurfile:79] Parsing Kurfile... [INFO 2017-02-28 22:28:49,038 kur.loggers.binary_logger:87] Log does not exist. Creating path: t3/cifar-log [INFO 2017-02-28 22:28:56,014 kur.providers.batch_provider:54] Batch size set to: 128 [INFO 2017-02-28 22:28:56,014 kur.providers.batch_provider:60] Maximum number of batches set to: 1000 [INFO 2017-02-28 22:29:02,413 kur.providers.batch_provider:54] Batch size set to: 32 [INFO 2017-02-28 22:29:02,413 kur.providers.batch_provider:60] Maximum number of batches set to: 50 [INFO 2017-02-28 22:29:02,414 kur.backend.backend:80] Creating backend: keras [INFO 2017-02-28 22:29:02,414 kur.backend.backend:83] Backend variants: none [INFO 2017-02-28 22:29:02,414 kur.backend.keras_backend:122] No particular backend for Keras has been requested. [INFO 2017-02-28 22:29:03,578 kur.backend.keras_backend:191] Keras is loaded. The backend is: theano [INFO 2017-02-28 22:29:03,579 kur.model.model:260] Enumerating the model containers. [INFO 2017-02-28 22:29:03,579 kur.model.model:265] Assembling the model dependency graph. [INFO 2017-02-28 22:29:03,579 kur.model.model:280] Connecting the model graph. [INFO 2017-02-28 22:29:04,579 kur.model.model:284] Model inputs: images [INFO 2017-02-28 22:29:04,579 kur.model.model:285] Model outputs: labels [INFO 2017-02-28 22:29:04,580 kur.kurfile:310] Ignoring missing initial weights: t3/cifar.best.valid.w. If this is undesireable, set "must_exist" to "yes" in the approriate "weights" section. [INFO 2017-02-28 22:29:05,910 kur.providers.batch_provider:54] Batch size set to: 2 [INFO 2017-02-28 22:29:05,910 kur.providers.batch_provider:60] Maximum number of batches set to: 1 [INFO 2017-02-28 22:29:05,913 kur.backend.keras_backend:654] Waiting for model to finish compiling... [INFO 2017-02-28 22:29:05,956 kur.model.executor:256] No historical training loss available from logs. [INFO 2017-02-28 22:29:05,956 kur.model.executor:264] No historical validation loss available from logs. [INFO 2017-02-28 22:29:05,956 kur.model.executor:270] No previous epochs. Epoch 1/20, loss=2.035: 100%|██████| 10000/10000 [00:02<00:00, 3344.74samples/s] [INFO 2017-02-28 22:29:08,954 kur.model.executor:390] Training loss: 2.035 [INFO 2017-02-28 22:29:08,954 kur.model.executor:397] Saving best historical training weights: t3/cifar.best.train.w [INFO 2017-02-28 22:29:09,202 kur.providers.batch_provider:54] Batch size set to: 2 [INFO 2017-02-28 22:29:09,203 kur.providers.batch_provider:60] Maximum number of batches set to: 1 [INFO 2017-02-28 22:29:09,208 kur.backend.keras_backend:654] Waiting for model to finish compiling... Validating, loss=1.854: 100%|████████| 1600/1600 [00:00<00:00, 4341.85samples/s] [INFO 2017-02-28 22:29:09,585 kur.model.executor:175] Validation loss: 1.854 [INFO 2017-02-28 22:29:09,585 kur.model.executor:422] Saving best historical validation weights: t3/cifar.best.valid.w Epoch 2/20, loss=1.693: 100%|██████| 10000/10000 [00:02<00:00, 4339.70samples/s] [INFO 2017-02-28 22:29:11,897 kur.model.executor:390] Training loss: 1.693 [INFO 2017-02-28 22:29:11,897 kur.model.executor:397] Saving best historical training weights: t3/cifar.best.train.w Validating, loss=1.641: 100%|████████| 1600/1600 [00:00<00:00, 7412.35samples/s] [INFO 2017-02-28 22:29:12,117 kur.model.executor:175] Validation loss: 1.641 [INFO 2017-02-28 22:29:12,118 kur.model.executor:422] Saving best historical validation weights: t3/cifar.best.valid.w Epoch 3/20, loss=1.541: 100%|██████| 10000/10000 [00:02<00:00, 4452.24samples/s] [INFO 2017-02-28 22:29:14,370 kur.model.executor:390] Training loss: 1.541 [INFO 2017-02-28 22:29:14,371 kur.model.executor:397] Saving best historical training weights: t3/cifar.best.train.w Validating, loss=1.548: 100%|████████| 1600/1600 [00:00<00:00, 8700.24samples/s] [INFO 2017-02-28 22:29:14,562 kur.model.executor:175] Validation loss: 1.548 [INFO 2017-02-28 22:29:14,562 kur.model.executor:422] Saving best historical validation weights: t3/cifar.best.valid.w Epoch 4/20, loss=1.444: 100%|██████| 10000/10000 [00:01<00:00, 5273.62samples/s] [INFO 2017-02-28 22:29:16,462 kur.model.executor:390] Training loss: 1.444 [INFO 2017-02-28 22:29:16,462 kur.model.executor:397] Saving best historical training weights: t3/cifar.best.train.w Validating, loss=1.510: 100%|████████| 1600/1600 [00:00<00:00, 8556.89samples/s] [INFO 2017-02-28 22:29:16,654 kur.model.executor:175] Validation loss: 1.510 [INFO 2017-02-28 22:29:16,654 kur.model.executor:422] Saving best historical validation weights: t3/cifar.best.valid.w Epoch 5/20, loss=1.376: 100%|██████| 10000/10000 [00:02<00:00, 4149.76samples/s] [INFO 2017-02-28 22:29:19,075 kur.model.executor:390] Training loss: 1.376 [INFO 2017-02-28 22:29:19,075 kur.model.executor:397] Saving best historical training weights: t3/cifar.best.train.w Validating, loss=1.431: 100%|████████| 1600/1600 [00:00<00:00, 4356.86samples/s] [INFO 2017-02-28 22:29:19,448 kur.model.executor:175] Validation loss: 1.431 [INFO 2017-02-28 22:29:19,448 kur.model.executor:422] Saving best historical validation weights: t3/cifar.best.valid.w Epoch 6/20, loss=1.319: 100%|██████| 10000/10000 [00:02<00:00, 3406.46samples/s] [INFO 2017-02-28 22:29:22,388 kur.model.executor:390] Training loss: 1.319 [INFO 2017-02-28 22:29:22,388 kur.model.executor:397] Saving best historical training weights: t3/cifar.best.train.w Validating, loss=1.411: 100%|████████| 1600/1600 [00:00<00:00, 4413.76samples/s] [INFO 2017-02-28 22:29:22,758 kur.model.executor:175] Validation loss: 1.411 [INFO 2017-02-28 22:29:22,758 kur.model.executor:422] Saving best historical validation weights: t3/cifar.best.valid.w Epoch 7/20, loss=1.265: 100%|██████| 10000/10000 [00:02<00:00, 4796.83samples/s] [INFO 2017-02-28 22:29:24,846 kur.model.executor:390] Training loss: 1.265 [INFO 2017-02-28 22:29:24,847 kur.model.executor:397] Saving best historical training weights: t3/cifar.best.train.w Validating, loss=1.365: 100%|████████| 1600/1600 [00:00<00:00, 5337.14samples/s] [INFO 2017-02-28 22:29:25,151 kur.model.executor:175] Validation loss: 1.365 [INFO 2017-02-28 22:29:25,151 kur.model.executor:422] Saving best historical validation weights: t3/cifar.best.valid.w Epoch 8/20, loss=1.226: 100%|██████| 10000/10000 [00:02<00:00, 4854.19samples/s] [INFO 2017-02-28 22:29:27,215 kur.model.executor:390] Training loss: 1.226 [INFO 2017-02-28 22:29:27,215 kur.model.executor:397] Saving best historical training weights: t3/cifar.best.train.w Validating, loss=1.401: 100%|████████| 1600/1600 [00:00<00:00, 6717.13samples/s] [INFO 2017-02-28 22:29:27,459 kur.model.executor:175] Validation loss: 1.401 Epoch 9/20, loss=1.189: 100%|██████| 10000/10000 [00:02<00:00, 4833.21samples/s] [INFO 2017-02-28 22:29:29,530 kur.model.executor:390] Training loss: 1.189 [INFO 2017-02-28 22:29:29,530 kur.model.executor:397] Saving best historical training weights: t3/cifar.best.train.w Validating, loss=1.325: 100%|████████| 1600/1600 [00:00<00:00, 6882.16samples/s] [INFO 2017-02-28 22:29:29,768 kur.model.executor:175] Validation loss: 1.325 [INFO 2017-02-28 22:29:29,769 kur.model.executor:422] Saving best historical validation weights: t3/cifar.best.valid.w Epoch 10/20, loss=1.155: 100%|█████| 10000/10000 [00:02<00:00, 4126.60samples/s] [INFO 2017-02-28 22:29:32,198 kur.model.executor:390] Training loss: 1.155 [INFO 2017-02-28 22:29:32,198 kur.model.executor:397] Saving best historical training weights: t3/cifar.best.train.w Validating, loss=1.275: 100%|████████| 1600/1600 [00:00<00:00, 6558.35samples/s] [INFO 2017-02-28 22:29:32,449 kur.model.executor:175] Validation loss: 1.275 [INFO 2017-02-28 22:29:32,449 kur.model.executor:422] Saving best historical validation weights: t3/cifar.best.valid.w Epoch 11/20, loss=1.124: 100%|█████| 10000/10000 [00:02<00:00, 4612.79samples/s] [INFO 2017-02-28 22:29:34,624 kur.model.executor:390] Training loss: 1.124 [INFO 2017-02-28 22:29:34,624 kur.model.executor:397] Saving best historical training weights: t3/cifar.best.train.w Validating, loss=1.297: 100%|████████| 1600/1600 [00:00<00:00, 7161.01samples/s] [INFO 2017-02-28 22:29:34,852 kur.model.executor:175] Validation loss: 1.297 Epoch 12/20, loss=1.093: 100%|█████| 10000/10000 [00:02<00:00, 4749.13samples/s] [INFO 2017-02-28 22:29:36,960 kur.model.executor:390] Training loss: 1.093 [INFO 2017-02-28 22:29:36,961 kur.model.executor:397] Saving best historical training weights: t3/cifar.best.train.w Validating, loss=1.298: 100%|████████| 1600/1600 [00:00<00:00, 6370.80samples/s] [INFO 2017-02-28 22:29:37,220 kur.model.executor:175] Validation loss: 1.298 Epoch 13/20, loss=1.064: 100%|█████| 10000/10000 [00:02<00:00, 3999.67samples/s] [INFO 2017-02-28 22:29:39,723 kur.model.executor:390] Training loss: 1.064 [INFO 2017-02-28 22:29:39,723 kur.model.executor:397] Saving best historical training weights: t3/cifar.best.train.w Validating, loss=1.282: 100%|████████| 1600/1600 [00:00<00:00, 6927.25samples/s] [INFO 2017-02-28 22:29:39,959 kur.model.executor:175] Validation loss: 1.282 Epoch 14/20, loss=1.049: 100%|█████| 10000/10000 [00:02<00:00, 4225.23samples/s] [INFO 2017-02-28 22:29:42,327 kur.model.executor:390] Training loss: 1.049 [INFO 2017-02-28 22:29:42,328 kur.model.executor:397] Saving best historical training weights: t3/cifar.best.train.w Validating, loss=1.285: 100%|████████| 1600/1600 [00:00<00:00, 4020.90samples/s] [INFO 2017-02-28 22:29:42,737 kur.model.executor:175] Validation loss: 1.285 Epoch 15/20, loss=1.029: 100%|█████| 10000/10000 [00:03<00:00, 3258.72samples/s] [INFO 2017-02-28 22:29:45,808 kur.model.executor:390] Training loss: 1.029 [INFO 2017-02-28 22:29:45,808 kur.model.executor:397] Saving best historical training weights: t3/cifar.best.train.w Validating, loss=1.313: 100%|████████| 1600/1600 [00:00<00:00, 3002.92samples/s] [INFO 2017-02-28 22:29:46,346 kur.model.executor:175] Validation loss: 1.313 Epoch 16/20, loss=1.000: 100%|█████| 10000/10000 [00:02<00:00, 3371.96samples/s] [INFO 2017-02-28 22:29:49,314 kur.model.executor:390] Training loss: 1.000 [INFO 2017-02-28 22:29:49,314 kur.model.executor:397] Saving best historical training weights: t3/cifar.best.train.w Validating, loss=1.244: 100%|████████| 1600/1600 [00:00<00:00, 4174.28samples/s] [INFO 2017-02-28 22:29:49,705 kur.model.executor:175] Validation loss: 1.244 [INFO 2017-02-28 22:29:49,706 kur.model.executor:422] Saving best historical validation weights: t3/cifar.best.valid.w Epoch 17/20, loss=0.986: 100%|█████| 10000/10000 [00:02<00:00, 4469.39samples/s] [INFO 2017-02-28 22:29:51,949 kur.model.executor:390] Training loss: 0.986 [INFO 2017-02-28 22:29:51,949 kur.model.executor:397] Saving best historical training weights: t3/cifar.best.train.w Validating, loss=1.294: 100%|████████| 1600/1600 [00:00<00:00, 8014.51samples/s] [INFO 2017-02-28 22:29:52,154 kur.model.executor:175] Validation loss: 1.294 Epoch 18/20, loss=0.968: 100%|█████| 10000/10000 [00:02<00:00, 4853.69samples/s] [INFO 2017-02-28 22:29:54,217 kur.model.executor:390] Training loss: 0.968 [INFO 2017-02-28 22:29:54,217 kur.model.executor:397] Saving best historical training weights: t3/cifar.best.train.w Validating, loss=1.352: 100%|████████| 1600/1600 [00:00<00:00, 7541.86samples/s] [INFO 2017-02-28 22:29:54,435 kur.model.executor:175] Validation loss: 1.352 Epoch 19/20, loss=0.963: 100%|█████| 10000/10000 [00:02<00:00, 4310.08samples/s] [INFO 2017-02-28 22:29:56,757 kur.model.executor:390] Training loss: 0.963 [INFO 2017-02-28 22:29:56,757 kur.model.executor:397] Saving best historical training weights: t3/cifar.best.train.w Validating, loss=1.215: 100%|████████| 1600/1600 [00:00<00:00, 5476.13samples/s] [INFO 2017-02-28 22:29:57,060 kur.model.executor:175] Validation loss: 1.215 [INFO 2017-02-28 22:29:57,060 kur.model.executor:422] Saving best historical validation weights: t3/cifar.best.valid.w Epoch 20/20, loss=0.930: 100%|█████| 10000/10000 [00:02<00:00, 4897.10samples/s] [INFO 2017-02-28 22:29:59,109 kur.model.executor:390] Training loss: 0.930 [INFO 2017-02-28 22:29:59,109 kur.model.executor:397] Saving best historical training weights: t3/cifar.best.train.w Validating, loss=1.259: 100%|████████| 1600/1600 [00:00<00:00, 8264.63samples/s] [INFO 2017-02-28 22:29:59,308 kur.model.executor:175] Validation loss: 1.259 Completed 20 epochs. [INFO 2017-02-28 22:29:59,309 kur.model.executor:210] Saving most recent weights: t3/cifar.last.w
# %cd t3/
!ls cifar-log
training_loss_labels validation_loss_labels training_loss_total validation_loss_total
from kur.loggers import BinaryLogger
training_loss = BinaryLogger.load_column('cifar-log', 'training_loss_total')
validation_loss = BinaryLogger.load_column('cifar-log', 'validation_loss_total')
training_loss
array([ 2.0347259 , 1.6929394 , 1.54147506, 1.44420362, 1.3758949 , 1.31863678, 1.26508343, 1.2262063 , 1.18933189, 1.15463436, 1.12429321, 1.09335482, 1.06431651, 1.04932415, 1.02948546, 0.99968618, 0.98606402, 0.9677121 , 0.96266526, 0.93041307], dtype=float32)
import matplotlib.pyplot as plt
plt.xlabel('Epoch')
plt.ylabel('Loss')
epoch = list(range(1, 1+len(training_loss)))
t_line, = plt.plot(epoch, training_loss, 'co-', label='Training Loss')
v_line, = plt.plot(epoch, validation_loss, 'mo-', label='Validation Loss')
plt.legend(handles=[t_line, v_line])
plt.show()
%%writefile dlnd_p2_dropout.yml
---
settings:
# Where to get the data
cifar: &cifar
url: "https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz"
checksum: "6d958be074577803d12ecdefd02955f39262c83c16fe9348329d7fe0b5c001ce"
path: "~/kur" # if kur does not have normalization or one-hot-encoding, ##################
# without normalization and one-hot-encoding, the performance will be hurt, right? ####
# Backend to use
backend:
name: keras
# Hyperparameters
cnn:
kernels: 20
size: [5, 5]
strides: [2, 2]
pool:
size: [2,2]
strides: [2,2]
type: max # must use a string here, {max} won't work, doc didn't say it #############
model:
- input: images # images are normalized from 0 to 1, labels are one-hot-encoding ##########
- convolution: # does kur do normalize and one-hot-encoding under the hood? ##############
kernels: "{{cnn.kernels}}"
size: "{{cnn.size}}"
strides: "{{ cnn.strides }}"
- activation: relu
- pool:
size: "{{pool.size}}"
strides: "{{pool.strides}}"
type: "{{pool.type}}"
- flatten:
- dense: 15 # p2 want a dropout applied here, but kur does not have yet?? ############
- dropout: 0.25
- dense: 10
- activation: softmax
name: labels
train:
data:
- cifar:
<<: *cifar
parts: [1] # only use dataset part 1 to train
provider:
batch_size: 128
num_batches: 1000 # the entire part 1 will be used
log: t1_dp0.25/cifar-log
epochs: 20
weights:
initial: t1_dp0.25/cifar.best.valid.w
best: t1_dp0.25/cifar.best.train.w
last: t1_dp0.25/cifar.last.w
optimizer:
name: adam
learning_rate: 0.001
validate:
data:
- cifar:
<<: *cifar
parts: 5 # only use dataset part 5 as validation set
provider:
batch_size: 128
num_batches: 50 # the project 2 only used 5000 data points as validation set
weights: t1_dp0.25/cifar.best.valid.w
test: &test
data:
- cifar:
<<: *cifar
parts: test
weights: t1_dp0.25/cifar.best.valid.w
provider:
batch_size: 128
num_batches: 1000 # the entire part test will be used
evaluate:
<<: *test
destination: t1_dp0.25/cifar.results.pkl
loss:
- target: labels # in the project: training loss and valid_accuracy are printed #############
name: categorical_crossentropy # this should be a matter of personal taste, won't really affect anything##
...
Overwriting dlnd_p2_dropout.yml
!kur -v train dlnd_p2_dropout.yml
[INFO 2017-03-01 07:34:45,517 kur.kurfile:699] Parsing source: dlnd_p2_dropout.yml, included by top-level. [INFO 2017-03-01 07:34:45,533 kur.kurfile:82] Parsing Kurfile... [INFO 2017-03-01 07:34:45,558 kur.loggers.binary_logger:107] Log does not exist. Creating path: t1_dp0.25/cifar-log [INFO 2017-03-01 07:34:58,394 kur.backend.backend:80] Creating backend: keras [INFO 2017-03-01 07:34:58,394 kur.backend.backend:83] Backend variants: none [INFO 2017-03-01 07:34:58,394 kur.backend.keras_backend:122] No particular backend for Keras has been requested. [INFO 2017-03-01 07:34:59,341 kur.backend.keras_backend:195] Keras is loaded. The backend is: theano [INFO 2017-03-01 07:34:59,341 kur.model.model:260] Enumerating the model containers. [INFO 2017-03-01 07:34:59,342 kur.model.model:265] Assembling the model dependency graph. [INFO 2017-03-01 07:34:59,342 kur.model.model:280] Connecting the model graph. [INFO 2017-03-01 07:35:00,404 kur.model.model:284] Model inputs: images [INFO 2017-03-01 07:35:00,404 kur.model.model:285] Model outputs: labels [INFO 2017-03-01 07:35:00,404 kur.kurfile:357] Ignoring missing initial weights: t1_dp0.25/cifar.best.valid.w. If this is undesireable, set "must_exist" to "yes" in the approriate "weights" section. [INFO 2017-03-01 07:35:00,404 kur.model.executor:315] No historical training loss available from logs. [INFO 2017-03-01 07:35:00,404 kur.model.executor:323] No historical validation loss available from logs. [INFO 2017-03-01 07:35:00,404 kur.model.executor:329] No previous epochs. [INFO 2017-03-01 07:35:01,829 kur.backend.keras_backend:666] Waiting for model to finish compiling... Epoch 1/20, loss=2.080: 100%|██████| 10000/10000 [00:02<00:00, 3728.18samples/s] [INFO 2017-03-01 07:35:04,562 kur.model.executor:464] Training loss: 2.080 [INFO 2017-03-01 07:35:04,562 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w [INFO 2017-03-01 07:35:04,953 kur.backend.keras_backend:666] Waiting for model to finish compiling... Validating, loss=1.874: 100%|███████| 6400/6400 [00:00<00:00, 10436.54samples/s] [INFO 2017-03-01 07:35:05,569 kur.model.executor:197] Validation loss: 1.874 [INFO 2017-03-01 07:35:05,570 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 2/20, loss=1.801: 100%|██████| 10000/10000 [00:02<00:00, 4985.77samples/s] [INFO 2017-03-01 07:35:07,580 kur.model.executor:464] Training loss: 1.801 [INFO 2017-03-01 07:35:07,580 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.672: 100%|███████| 6400/6400 [00:00<00:00, 10588.70samples/s] [INFO 2017-03-01 07:35:08,196 kur.model.executor:197] Validation loss: 1.672 [INFO 2017-03-01 07:35:08,196 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 3/20, loss=1.646: 100%|██████| 10000/10000 [00:02<00:00, 4988.95samples/s] [INFO 2017-03-01 07:35:10,206 kur.model.executor:464] Training loss: 1.646 [INFO 2017-03-01 07:35:10,206 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.578: 100%|███████| 6400/6400 [00:00<00:00, 11537.31samples/s] [INFO 2017-03-01 07:35:10,768 kur.model.executor:197] Validation loss: 1.578 [INFO 2017-03-01 07:35:10,768 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 4/20, loss=1.537: 100%|██████| 10000/10000 [00:02<00:00, 4735.37samples/s] [INFO 2017-03-01 07:35:12,886 kur.model.executor:464] Training loss: 1.537 [INFO 2017-03-01 07:35:12,887 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.490: 100%|███████| 6400/6400 [00:00<00:00, 10859.18samples/s] [INFO 2017-03-01 07:35:13,486 kur.model.executor:197] Validation loss: 1.490 [INFO 2017-03-01 07:35:13,486 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 5/20, loss=1.470: 100%|██████| 10000/10000 [00:02<00:00, 4768.44samples/s] [INFO 2017-03-01 07:35:15,588 kur.model.executor:464] Training loss: 1.470 [INFO 2017-03-01 07:35:15,588 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.411: 100%|███████| 6400/6400 [00:00<00:00, 10774.77samples/s] [INFO 2017-03-01 07:35:16,189 kur.model.executor:197] Validation loss: 1.411 [INFO 2017-03-01 07:35:16,190 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 6/20, loss=1.406: 100%|██████| 10000/10000 [00:02<00:00, 4293.21samples/s] [INFO 2017-03-01 07:35:18,523 kur.model.executor:464] Training loss: 1.406 [INFO 2017-03-01 07:35:18,524 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.384: 100%|███████| 6400/6400 [00:00<00:00, 10308.95samples/s] [INFO 2017-03-01 07:35:19,153 kur.model.executor:197] Validation loss: 1.384 [INFO 2017-03-01 07:35:19,154 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 7/20, loss=1.371: 100%|██████| 10000/10000 [00:02<00:00, 4549.75samples/s] [INFO 2017-03-01 07:35:21,358 kur.model.executor:464] Training loss: 1.371 [INFO 2017-03-01 07:35:21,358 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.368: 100%|████████| 6400/6400 [00:00<00:00, 4923.30samples/s] [INFO 2017-03-01 07:35:21,993 kur.model.executor:197] Validation loss: 1.368 [INFO 2017-03-01 07:35:21,994 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 8/20, loss=1.330: 100%|██████| 10000/10000 [00:02<00:00, 4738.82samples/s] [INFO 2017-03-01 07:35:24,111 kur.model.executor:464] Training loss: 1.330 [INFO 2017-03-01 07:35:24,111 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.328: 100%|███████| 6400/6400 [00:00<00:00, 10351.03samples/s] [INFO 2017-03-01 07:35:24,737 kur.model.executor:197] Validation loss: 1.328 [INFO 2017-03-01 07:35:24,738 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 9/20, loss=1.303: 100%|██████| 10000/10000 [00:02<00:00, 4794.48samples/s] [INFO 2017-03-01 07:35:26,830 kur.model.executor:464] Training loss: 1.303 [INFO 2017-03-01 07:35:26,830 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.339: 100%|███████| 6400/6400 [00:00<00:00, 11131.73samples/s] [INFO 2017-03-01 07:35:27,412 kur.model.executor:197] Validation loss: 1.339 Epoch 10/20, loss=1.294: 100%|█████| 10000/10000 [00:02<00:00, 4693.60samples/s] [INFO 2017-03-01 07:35:29,546 kur.model.executor:464] Training loss: 1.294 [INFO 2017-03-01 07:35:29,546 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.310: 100%|███████| 6400/6400 [00:00<00:00, 11321.00samples/s] [INFO 2017-03-01 07:35:30,118 kur.model.executor:197] Validation loss: 1.310 [INFO 2017-03-01 07:35:30,119 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 11/20, loss=1.253: 100%|█████| 10000/10000 [00:02<00:00, 4719.59samples/s] [INFO 2017-03-01 07:35:32,243 kur.model.executor:464] Training loss: 1.253 [INFO 2017-03-01 07:35:32,243 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.293: 100%|███████| 6400/6400 [00:00<00:00, 10417.43samples/s] [INFO 2017-03-01 07:35:32,866 kur.model.executor:197] Validation loss: 1.293 [INFO 2017-03-01 07:35:32,867 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 12/20, loss=1.233: 100%|█████| 10000/10000 [00:02<00:00, 4647.94samples/s] [INFO 2017-03-01 07:35:35,024 kur.model.executor:464] Training loss: 1.233 [INFO 2017-03-01 07:35:35,024 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.268: 100%|████████| 6400/6400 [00:00<00:00, 9125.34samples/s] [INFO 2017-03-01 07:35:35,748 kur.model.executor:197] Validation loss: 1.268 [INFO 2017-03-01 07:35:35,748 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 13/20, loss=1.213: 100%|█████| 10000/10000 [00:02<00:00, 4052.58samples/s] [INFO 2017-03-01 07:35:38,221 kur.model.executor:464] Training loss: 1.213 [INFO 2017-03-01 07:35:38,221 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.263: 100%|███████| 6400/6400 [00:00<00:00, 10334.80samples/s] [INFO 2017-03-01 07:35:38,848 kur.model.executor:197] Validation loss: 1.263 [INFO 2017-03-01 07:35:38,848 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 14/20, loss=1.192: 100%|█████| 10000/10000 [00:02<00:00, 4059.79samples/s] [INFO 2017-03-01 07:35:41,316 kur.model.executor:464] Training loss: 1.192 [INFO 2017-03-01 07:35:41,316 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.267: 100%|███████| 6400/6400 [00:00<00:00, 10624.42samples/s] [INFO 2017-03-01 07:35:41,927 kur.model.executor:197] Validation loss: 1.267 Epoch 15/20, loss=1.179: 100%|█████| 10000/10000 [00:03<00:00, 3174.28samples/s] [INFO 2017-03-01 07:35:45,081 kur.model.executor:464] Training loss: 1.179 [INFO 2017-03-01 07:35:45,081 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.250: 100%|███████| 6400/6400 [00:00<00:00, 10746.89samples/s] [INFO 2017-03-01 07:35:45,686 kur.model.executor:197] Validation loss: 1.250 [INFO 2017-03-01 07:35:45,686 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 16/20, loss=1.155: 100%|█████| 10000/10000 [00:02<00:00, 3838.08samples/s] [INFO 2017-03-01 07:35:48,298 kur.model.executor:464] Training loss: 1.155 [INFO 2017-03-01 07:35:48,298 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.249: 100%|███████| 6400/6400 [00:00<00:00, 10454.68samples/s] [INFO 2017-03-01 07:35:48,938 kur.model.executor:197] Validation loss: 1.249 [INFO 2017-03-01 07:35:48,939 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 17/20, loss=1.141: 100%|█████| 10000/10000 [00:02<00:00, 3516.54samples/s] [INFO 2017-03-01 07:35:51,791 kur.model.executor:464] Training loss: 1.141 [INFO 2017-03-01 07:35:51,791 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.235: 100%|███████| 6400/6400 [00:00<00:00, 10439.16samples/s] [INFO 2017-03-01 07:35:52,412 kur.model.executor:197] Validation loss: 1.235 [INFO 2017-03-01 07:35:52,412 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 18/20, loss=1.123: 100%|█████| 10000/10000 [00:03<00:00, 3276.45samples/s] [INFO 2017-03-01 07:35:55,469 kur.model.executor:464] Training loss: 1.123 [INFO 2017-03-01 07:35:55,470 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.238: 100%|████████| 6400/6400 [00:00<00:00, 7017.06samples/s] [INFO 2017-03-01 07:35:56,390 kur.model.executor:197] Validation loss: 1.238 Epoch 19/20, loss=1.121: 100%|█████| 10000/10000 [00:02<00:00, 4330.83samples/s] [INFO 2017-03-01 07:35:58,703 kur.model.executor:464] Training loss: 1.121 [INFO 2017-03-01 07:35:58,704 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.224: 100%|███████| 6400/6400 [00:00<00:00, 11403.63samples/s] [INFO 2017-03-01 07:35:59,272 kur.model.executor:197] Validation loss: 1.224 [INFO 2017-03-01 07:35:59,272 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 20/20, loss=1.107: 100%|█████| 10000/10000 [00:02<00:00, 4730.36samples/s] [INFO 2017-03-01 07:36:01,393 kur.model.executor:464] Training loss: 1.107 [INFO 2017-03-01 07:36:01,394 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.234: 100%|███████| 6400/6400 [00:00<00:00, 11209.60samples/s] [INFO 2017-03-01 07:36:01,972 kur.model.executor:197] Validation loss: 1.234 Completed 20 epochs. [INFO 2017-03-01 07:36:01,976 kur.model.executor:235] Saving most recent weights: t1_dp0.25/cifar.last.w
# %cd t1_dp0.25/
!ls cifar-log
batch_loss_batch training_loss_batch validation_loss_labels batch_loss_labels training_loss_labels validation_loss_total batch_loss_total training_loss_total summary.yml validation_loss_batch
%pycat summary.yml
ERROR:root:Cell magic `%%pycat` not found (But line magic `%pycat` exists, did you mean that instead?).
from kur.loggers import BinaryLogger
training_loss = BinaryLogger.load_column('cifar-log', 'training_loss_total')
validation_loss = BinaryLogger.load_column('cifar-log', 'validation_loss_total')
training_loss
array([ 2.08002758, 1.80103576, 1.64649773, 1.53694201, 1.47005808, 1.4060986 , 1.37083662, 1.32995439, 1.3027122 , 1.29357958, 1.25282907, 1.23259354, 1.21278262, 1.19201612, 1.17865086, 1.15514135, 1.14136755, 1.1234889 , 1.1214298 , 1.10675633], dtype=float32)
import matplotlib.pyplot as plt
plt.xlabel('Epoch')
plt.ylabel('Loss')
epoch = list(range(1, 1+len(training_loss)))
t_line, = plt.plot(epoch, training_loss, 'co-', label='Training Loss')
v_line, = plt.plot(epoch, validation_loss, 'mo-', label='Validation Loss')
plt.legend(handles=[t_line, v_line])
plt.show()
Task1: Split kurfile
Solution1: Split kurfile
Create a dlnd_p2_defaults.yml
%%writefile dlnd_p2_defaults.yml
---
settings:
# Where to get the data
cifar: &cifar
url: "https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz"
checksum: "6d958be074577803d12ecdefd02955f39262c83c16fe9348329d7fe0b5c001ce"
path: "~/kur"
# Backend to use
backend:
name: keras
model: # there should be loop design for model structure flexibility
- input: images
- convolution:
kernels: "{{cnn.kernels}}"
size: "{{cnn.size}}"
strides: "{{ cnn.strides }}"
- activation: relu
- pool:
size: "{{pool.size}}"
strides: "{{pool.strides}}"
type: "{{pool.type}}"
- flatten:
- dense: "{{dense.out1}}" # 15
- dense: "{{dense.out2}}" # 10
- activation: softmax
name: labels
train:
data:
- cifar:
<<: *cifar
parts: "{{part.train}}" # only use dataset part 1 to train
provider:
batch_size: "{{batch_size.train}}" # 128
num_batches: "{{num_batches.train}}" # total batch is less than 100, so use 1000 is to select all #####
randomize: True # Importance: when train on small sample we still want weights to have
# generalize power. model overfit by memorizing the same data points####
log: "{{path.log}}" # t1_dp0.25/cifar-log
epochs: "{{num_epochs}}" # 20
weights:
initial: "{{path.initial_w}}" # t1_dp0.25/cifar.best.valid.w
best: "{{path.best_train_w}}" # t1_dp0.25/cifar.best.train.w
last: "{{path.last_w}}" # t1_dp0.25/cifar.last.w
optimizer:
name: adam
learning_rate: "{{learning_rate}}" # 0.001
validate:
data:
- cifar:
<<: *cifar
parts: "{{part.valid}}" # only use dataset part 5 as validation set
provider:
batch_size: "{{batch_size.valid}}" # same as training, 128
num_batches: "{{num_batches.valid}}" # the project 2 only used 5000 data points as validation set, so 50
weights: "{{path.best_valid_w}}" # t1_dp0.25/cifar.best.valid.w
test: &test
data:
- cifar:
<<: *cifar
parts: test
weights: "{{path.best_valid_w}}" # t1_dp0.25/cifar.best.valid.w
provider:
batch_size: "{{batch_size.test}}" # same as training, 128
num_batches: "{{num_batches.test}}" # the entire part test will be used, set it 1000
evaluate:
<<: *test
destination: "{{path.result}}" # t1_dp0.25/cifar.results.pkl
loss:
- target: labels
name: categorical_crossentropy
...
Overwriting dlnd_p2_defaults.yml
Create dlnd_p2_fluid.yml
%%writefile dlnd_p2_fluid.yml
---
settings:
cnn:
kernels: 20 # why cannot use [20]? what if there are 2 conv layers [20, 30]
size: [5,5]
strides: [2,2]
pool:
size: [2,2]
strides: [2,2]
type: "max"
dense:
out1: 15
out2: 10
part:
train: 1
valid: 5
test: test
batch_size:
train: 128
valid: 128
test: 128
num_batches:
train: # 1000 and nothing here both refer to all the batches? ###########################
valid: 50
test:
path:
log: t1_dp0.25/cifar-log
initial_w: t1_dp0.25/cifar.best.valid.w
best_train_w: t1_dp0.25/cifar.best.train.w
last_w: t1_dp0.25/cifar.last.w
best_valid_w: t1_dp0.25/cifar.best.valid.w
result: t1_dp0.25/cifar.results.pkl
learning_rate: 0.001
num_epochs: 1
include: dlnd_p2_defaults.yml # to define default setting
...
Overwriting dlnd_p2_fluid.yml
!kur -vv train dlnd_p2_fluid.yml
[INFO 2017-03-01 15:24:19,985 kur.kurfile:699] Parsing source: dlnd_p2_fluid.yml, included by top-level. [INFO 2017-03-01 15:24:19,996 kur.kurfile:699] Parsing source: dlnd_p2_defaults.yml, included by dlnd_p2_fluid.yml. [INFO 2017-03-01 15:24:20,010 kur.kurfile:82] Parsing Kurfile... [DEBUG 2017-03-01 15:24:20,010 kur.kurfile:784] Parsing Kurfile section: settings [DEBUG 2017-03-01 15:24:20,016 kur.kurfile:784] Parsing Kurfile section: train [DEBUG 2017-03-01 15:24:20,023 kur.kurfile:784] Parsing Kurfile section: validate [DEBUG 2017-03-01 15:24:20,028 kur.kurfile:784] Parsing Kurfile section: test [DEBUG 2017-03-01 15:24:20,033 kur.kurfile:784] Parsing Kurfile section: evaluate [DEBUG 2017-03-01 15:24:20,039 kur.containers.layers.placeholder:63] Using short-hand name for placeholder: images [DEBUG 2017-03-01 15:24:20,040 kur.containers.layers.placeholder:97] Placeholder "images" has a deferred shape. [DEBUG 2017-03-01 15:24:20,050 kur.kurfile:784] Parsing Kurfile section: loss [INFO 2017-03-01 15:24:20,052 kur.loggers.binary_logger:107] Log does not exist. Creating path: t1_dp0.25/cifar-log [DEBUG 2017-03-01 15:24:20,980 kur.utils.package:233] File exists and passed checksum: /Users/Natsume/kur/cifar-10-python.tar.gz [DEBUG 2017-03-01 15:24:27,155 kur.providers.batch_provider:57] Batch size set to: 128 [DEBUG 2017-03-01 15:24:27,863 kur.utils.package:233] File exists and passed checksum: /Users/Natsume/kur/cifar-10-python.tar.gz [DEBUG 2017-03-01 15:24:33,548 kur.providers.batch_provider:57] Batch size set to: 128 [DEBUG 2017-03-01 15:24:33,548 kur.providers.batch_provider:102] Maximum number of batches set to: 50 [DEBUG 2017-03-01 15:24:33,548 kur.backend.backend:187] Using backend: keras [INFO 2017-03-01 15:24:33,549 kur.backend.backend:80] Creating backend: keras [INFO 2017-03-01 15:24:33,549 kur.backend.backend:83] Backend variants: none [INFO 2017-03-01 15:24:33,549 kur.backend.keras_backend:122] No particular backend for Keras has been requested. [DEBUG 2017-03-01 15:24:33,549 kur.backend.keras_backend:124] Using the system-default Keras backend. [DEBUG 2017-03-01 15:24:33,549 kur.backend.keras_backend:189] Overriding environmental variables: {'TF_CPP_MIN_LOG_LEVEL': '1', 'KERAS_BACKEND': None, 'THEANO_FLAGS': None} [INFO 2017-03-01 15:24:34,503 kur.backend.keras_backend:195] Keras is loaded. The backend is: theano [INFO 2017-03-01 15:24:34,503 kur.model.model:260] Enumerating the model containers. [INFO 2017-03-01 15:24:34,503 kur.model.model:265] Assembling the model dependency graph. [DEBUG 2017-03-01 15:24:34,503 kur.model.model:272] Assembled Node: images [DEBUG 2017-03-01 15:24:34,503 kur.model.model:274] Uses: [DEBUG 2017-03-01 15:24:34,503 kur.model.model:276] Used by: ..convolution.0 [DEBUG 2017-03-01 15:24:34,503 kur.model.model:277] Aliases: images [DEBUG 2017-03-01 15:24:34,503 kur.model.model:272] Assembled Node: ..convolution.0 [DEBUG 2017-03-01 15:24:34,503 kur.model.model:274] Uses: images [DEBUG 2017-03-01 15:24:34,503 kur.model.model:276] Used by: ..activation.0 [DEBUG 2017-03-01 15:24:34,503 kur.model.model:277] Aliases: ..convolution.0 [DEBUG 2017-03-01 15:24:34,504 kur.model.model:272] Assembled Node: ..activation.0 [DEBUG 2017-03-01 15:24:34,504 kur.model.model:274] Uses: ..convolution.0 [DEBUG 2017-03-01 15:24:34,504 kur.model.model:276] Used by: ..pool.0 [DEBUG 2017-03-01 15:24:34,504 kur.model.model:277] Aliases: ..activation.0 [DEBUG 2017-03-01 15:24:34,504 kur.model.model:272] Assembled Node: ..pool.0 [DEBUG 2017-03-01 15:24:34,504 kur.model.model:274] Uses: ..activation.0 [DEBUG 2017-03-01 15:24:34,504 kur.model.model:276] Used by: ..flatten.0 [DEBUG 2017-03-01 15:24:34,504 kur.model.model:277] Aliases: ..pool.0 [DEBUG 2017-03-01 15:24:34,504 kur.model.model:272] Assembled Node: ..flatten.0 [DEBUG 2017-03-01 15:24:34,504 kur.model.model:274] Uses: ..pool.0 [DEBUG 2017-03-01 15:24:34,504 kur.model.model:276] Used by: ..dense.0 [DEBUG 2017-03-01 15:24:34,504 kur.model.model:277] Aliases: ..flatten.0 [DEBUG 2017-03-01 15:24:34,504 kur.model.model:272] Assembled Node: ..dense.0 [DEBUG 2017-03-01 15:24:34,504 kur.model.model:274] Uses: ..flatten.0 [DEBUG 2017-03-01 15:24:34,504 kur.model.model:276] Used by: ..dense.1 [DEBUG 2017-03-01 15:24:34,504 kur.model.model:277] Aliases: ..dense.0 [DEBUG 2017-03-01 15:24:34,504 kur.model.model:272] Assembled Node: ..dense.1 [DEBUG 2017-03-01 15:24:34,504 kur.model.model:274] Uses: ..dense.0 [DEBUG 2017-03-01 15:24:34,504 kur.model.model:276] Used by: labels [DEBUG 2017-03-01 15:24:34,504 kur.model.model:277] Aliases: ..dense.1 [DEBUG 2017-03-01 15:24:34,504 kur.model.model:272] Assembled Node: labels [DEBUG 2017-03-01 15:24:34,505 kur.model.model:274] Uses: ..dense.1 [DEBUG 2017-03-01 15:24:34,505 kur.model.model:276] Used by: [DEBUG 2017-03-01 15:24:34,505 kur.model.model:277] Aliases: labels [INFO 2017-03-01 15:24:34,505 kur.model.model:280] Connecting the model graph. [DEBUG 2017-03-01 15:24:34,505 kur.model.model:311] Building node: images [DEBUG 2017-03-01 15:24:34,505 kur.model.model:312] Aliases: images [DEBUG 2017-03-01 15:24:34,505 kur.model.model:313] Inputs: [DEBUG 2017-03-01 15:24:34,505 kur.containers.layers.placeholder:117] Creating placeholder for "images" with data type "float32". [DEBUG 2017-03-01 15:24:34,505 kur.model.model:125] Trying to infer shape for input "images" [DEBUG 2017-03-01 15:24:34,505 kur.model.model:143] Inferred shape for input "images": (32, 32, 3) [DEBUG 2017-03-01 15:24:34,505 kur.containers.layers.placeholder:127] Inferred shape: (32, 32, 3) [DEBUG 2017-03-01 15:24:34,510 kur.model.model:382] Value: images [DEBUG 2017-03-01 15:24:34,510 kur.model.model:311] Building node: ..convolution.0 [DEBUG 2017-03-01 15:24:34,510 kur.model.model:312] Aliases: ..convolution.0 [DEBUG 2017-03-01 15:24:34,510 kur.model.model:313] Inputs: [DEBUG 2017-03-01 15:24:34,510 kur.model.model:315] - images: images [DEBUG 2017-03-01 15:24:35,577 kur.model.model:382] Value: Elemwise{add,no_inplace}.0 [DEBUG 2017-03-01 15:24:35,577 kur.model.model:311] Building node: ..activation.0 [DEBUG 2017-03-01 15:24:35,577 kur.model.model:312] Aliases: ..activation.0 [DEBUG 2017-03-01 15:24:35,577 kur.model.model:313] Inputs: [DEBUG 2017-03-01 15:24:35,577 kur.model.model:315] - ..convolution.0: Elemwise{add,no_inplace}.0 [DEBUG 2017-03-01 15:24:35,612 kur.model.model:382] Value: Elemwise{mul,no_inplace}.0 [DEBUG 2017-03-01 15:24:35,612 kur.model.model:311] Building node: ..pool.0 [DEBUG 2017-03-01 15:24:35,612 kur.model.model:312] Aliases: ..pool.0 [DEBUG 2017-03-01 15:24:35,612 kur.model.model:313] Inputs: [DEBUG 2017-03-01 15:24:35,612 kur.model.model:315] - ..activation.0: Elemwise{mul,no_inplace}.0 [DEBUG 2017-03-01 15:24:35,615 kur.model.model:382] Value: DimShuffle{0,2,3,1}.0 [DEBUG 2017-03-01 15:24:35,615 kur.model.model:311] Building node: ..flatten.0 [DEBUG 2017-03-01 15:24:35,615 kur.model.model:312] Aliases: ..flatten.0 [DEBUG 2017-03-01 15:24:35,615 kur.model.model:313] Inputs: [DEBUG 2017-03-01 15:24:35,615 kur.model.model:315] - ..pool.0: DimShuffle{0,2,3,1}.0 [DEBUG 2017-03-01 15:24:35,622 kur.model.model:382] Value: Reshape{2}.0 [DEBUG 2017-03-01 15:24:35,622 kur.model.model:311] Building node: ..dense.0 [DEBUG 2017-03-01 15:24:35,622 kur.model.model:312] Aliases: ..dense.0 [DEBUG 2017-03-01 15:24:35,622 kur.model.model:313] Inputs: [DEBUG 2017-03-01 15:24:35,622 kur.model.model:315] - ..flatten.0: Reshape{2}.0 [DEBUG 2017-03-01 15:24:35,624 kur.model.model:382] Value: Elemwise{add,no_inplace}.0 [DEBUG 2017-03-01 15:24:35,624 kur.model.model:311] Building node: ..dense.1 [DEBUG 2017-03-01 15:24:35,624 kur.model.model:312] Aliases: ..dense.1 [DEBUG 2017-03-01 15:24:35,624 kur.model.model:313] Inputs: [DEBUG 2017-03-01 15:24:35,624 kur.model.model:315] - ..dense.0: Elemwise{add,no_inplace}.0 [DEBUG 2017-03-01 15:24:35,625 kur.model.model:382] Value: Elemwise{add,no_inplace}.0 [DEBUG 2017-03-01 15:24:35,625 kur.model.model:311] Building node: labels [DEBUG 2017-03-01 15:24:35,625 kur.model.model:312] Aliases: labels [DEBUG 2017-03-01 15:24:35,625 kur.model.model:313] Inputs: [DEBUG 2017-03-01 15:24:35,625 kur.model.model:315] - ..dense.1: Elemwise{add,no_inplace}.0 [DEBUG 2017-03-01 15:24:35,625 kur.model.model:382] Value: Softmax.0 [INFO 2017-03-01 15:24:35,625 kur.model.model:284] Model inputs: images [INFO 2017-03-01 15:24:35,625 kur.model.model:285] Model outputs: labels [INFO 2017-03-01 15:24:35,625 kur.kurfile:357] Ignoring missing initial weights: t1_dp0.25/cifar.best.valid.w. If this is undesireable, set "must_exist" to "yes" in the approriate "weights" section. [INFO 2017-03-01 15:24:35,625 kur.model.executor:315] No historical training loss available from logs. [INFO 2017-03-01 15:24:35,626 kur.model.executor:323] No historical validation loss available from logs. [INFO 2017-03-01 15:24:35,626 kur.model.executor:329] No previous epochs. [DEBUG 2017-03-01 15:24:35,626 kur.model.executor:353] Epoch handling mode: additional [DEBUG 2017-03-01 15:24:35,626 kur.model.executor:101] Recompiling the model. [DEBUG 2017-03-01 15:24:35,626 kur.backend.keras_backend:527] Instantiating a Keras model. [DEBUG 2017-03-01 15:24:35,825 kur.backend.keras_backend:538] ____________________________________________________________________________________________________ [DEBUG 2017-03-01 15:24:35,825 kur.backend.keras_backend:538] Layer (type) Output Shape Param # Connected to [DEBUG 2017-03-01 15:24:35,825 kur.backend.keras_backend:538] ==================================================================================================== [DEBUG 2017-03-01 15:24:35,825 kur.backend.keras_backend:538] images (InputLayer) (None, 32, 32, 3) 0 [DEBUG 2017-03-01 15:24:35,825 kur.backend.keras_backend:538] ____________________________________________________________________________________________________ [DEBUG 2017-03-01 15:24:35,825 kur.backend.keras_backend:538] ..convolution.0 (Convolution2D) (None, 16, 16, 20) 1520 images[0][0] [DEBUG 2017-03-01 15:24:35,825 kur.backend.keras_backend:538] ____________________________________________________________________________________________________ [DEBUG 2017-03-01 15:24:35,826 kur.backend.keras_backend:538] ..activation.0 (Activation) (None, 16, 16, 20) 0 ..convolution.0[0][0] [DEBUG 2017-03-01 15:24:35,826 kur.backend.keras_backend:538] ____________________________________________________________________________________________________ [DEBUG 2017-03-01 15:24:35,826 kur.backend.keras_backend:538] ..pool.0 (MaxPooling2D) (None, 8, 8, 20) 0 ..activation.0[0][0] [DEBUG 2017-03-01 15:24:35,826 kur.backend.keras_backend:538] ____________________________________________________________________________________________________ [DEBUG 2017-03-01 15:24:35,826 kur.backend.keras_backend:538] ..flatten.0 (Flatten) (None, 1280) 0 ..pool.0[0][0] [DEBUG 2017-03-01 15:24:35,826 kur.backend.keras_backend:538] ____________________________________________________________________________________________________ [DEBUG 2017-03-01 15:24:35,826 kur.backend.keras_backend:538] ..dense.0 (Dense) (None, 15) 19215 ..flatten.0[0][0] [DEBUG 2017-03-01 15:24:35,826 kur.backend.keras_backend:538] ____________________________________________________________________________________________________ [DEBUG 2017-03-01 15:24:35,826 kur.backend.keras_backend:538] ..dense.1 (Dense) (None, 10) 160 ..dense.0[0][0] [DEBUG 2017-03-01 15:24:35,826 kur.backend.keras_backend:538] ____________________________________________________________________________________________________ [DEBUG 2017-03-01 15:24:35,826 kur.backend.keras_backend:538] labels (Activation) (None, 10) 0 ..dense.1[0][0] [DEBUG 2017-03-01 15:24:35,826 kur.backend.keras_backend:538] ==================================================================================================== [DEBUG 2017-03-01 15:24:35,826 kur.backend.keras_backend:538] Total params: 20,895 [DEBUG 2017-03-01 15:24:35,826 kur.backend.keras_backend:538] Trainable params: 20,895 [DEBUG 2017-03-01 15:24:35,826 kur.backend.keras_backend:538] Non-trainable params: 0 [DEBUG 2017-03-01 15:24:35,826 kur.backend.keras_backend:538] ____________________________________________________________________________________________________ [DEBUG 2017-03-01 15:24:35,826 kur.backend.keras_backend:538] [DEBUG 2017-03-01 15:24:35,826 kur.backend.keras_backend:576] Assembling a training function from the model. [DEBUG 2017-03-01 15:24:35,831 kur.backend.keras_backend:509] Adding additional inputs: labels [DEBUG 2017-03-01 15:24:37,417 kur.backend.keras_backend:599] Additional inputs for log functions: labels [DEBUG 2017-03-01 15:24:37,417 kur.backend.keras_backend:616] Expected input shapes: images=(None, 32, 32, 3), labels=(None, None) [DEBUG 2017-03-01 15:24:37,417 kur.backend.keras_backend:634] Compiled model: {'names': {'output': ['labels', 'labels'], 'input': ['images', 'labels']}, 'func': <keras.backend.theano_backend.Function object at 0x129e70ef0>, 'shapes': {'input': [(None, 32, 32, 3), (None, None)]}} [DEBUG 2017-03-01 15:24:37,417 kur.providers.batch_provider:57] Batch size set to: 2 [DEBUG 2017-03-01 15:24:37,417 kur.providers.batch_provider:102] Maximum number of batches set to: 1 [INFO 2017-03-01 15:24:37,420 kur.backend.keras_backend:666] Waiting for model to finish compiling... [DEBUG 2017-03-01 15:24:37,420 kur.providers.batch_provider:139] Preparing next batch of data... [DEBUG 2017-03-01 15:24:37,420 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:37,469 kur.providers.provider:144] Data source "labels": entries=10000, shape=(10,) [DEBUG 2017-03-01 15:24:37,470 kur.providers.provider:144] Data source "images": entries=10000, shape=(32, 32, 3) Epoch 1/1, loss=N/A: 0%| | 0/10000 [00:00<?, ?samples/s][DEBUG 2017-03-01 15:24:37,475 kur.providers.shuffle_provider:184] Shuffling... [DEBUG 2017-03-01 15:24:37,646 kur.providers.batch_provider:139] Preparing next batch of data... [DEBUG 2017-03-01 15:24:37,647 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:37,647 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:37,656 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:37,681 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:37,682 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:37,682 kur.loggers.binary_logger:135] Adding data to binary column: batch_loss_total [DEBUG 2017-03-01 15:24:37,683 kur.loggers.binary_logger:135] Adding data to binary column: batch_loss_labels [DEBUG 2017-03-01 15:24:37,683 kur.loggers.binary_logger:135] Adding data to binary column: batch_loss_batch [DEBUG 2017-03-01 15:24:37,683 kur.loggers.binary_logger:144] Writing logger summary. Epoch 1/1, loss=2.321: 1%|▏ | 128/10000 [00:00<00:16, 610.76samples/s][DEBUG 2017-03-01 15:24:37,685 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:37,694 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:37,698 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:37,717 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:37,717 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:37,718 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:37,730 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:37,754 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:37,755 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:37,755 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:37,772 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:37,797 kur.model.executor:597] Finished training on batch. Epoch 1/1, loss=2.325: 5%|▌ | 512/10000 [00:00<00:11, 810.28samples/s][DEBUG 2017-03-01 15:24:37,798 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:37,798 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:37,813 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:37,829 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:37,830 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:37,838 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:37,844 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:37,868 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:37,869 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:37,869 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:37,882 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:37,902 kur.model.executor:597] Finished training on batch. Epoch 1/1, loss=2.314: 9%|▊ | 896/10000 [00:00<00:08, 1057.23samples/s][DEBUG 2017-03-01 15:24:37,902 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:37,903 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:37,916 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:37,938 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:37,940 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:37,951 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:37,957 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:37,977 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:37,977 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:37,977 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:37,990 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,011 kur.model.executor:597] Finished training on batch. Epoch 1/1, loss=2.304: 13%|█ | 1280/10000 [00:00<00:06, 1338.00samples/s][DEBUG 2017-03-01 15:24:38,012 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,012 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,028 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,046 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:38,047 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,047 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,063 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,086 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:38,086 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,086 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,097 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,127 kur.model.executor:597] Finished training on batch. Epoch 1/1, loss=2.287: 17%|█▎ | 1664/10000 [00:00<00:05, 1630.08samples/s][DEBUG 2017-03-01 15:24:38,127 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,127 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,137 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,164 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:38,165 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,165 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,178 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,199 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:38,199 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,200 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,210 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,234 kur.model.executor:597] Finished training on batch. Epoch 1/1, loss=2.277: 20%|█▋ | 2048/10000 [00:00<00:04, 1947.87samples/s][DEBUG 2017-03-01 15:24:38,235 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,235 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,244 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,270 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:38,270 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,270 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,278 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,301 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:38,302 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,302 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,311 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,333 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:38,334 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,334 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,346 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,370 kur.model.executor:597] Finished training on batch. Epoch 1/1, loss=2.261: 26%|██ | 2560/10000 [00:00<00:03, 2278.82samples/s][DEBUG 2017-03-01 15:24:38,370 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,371 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,384 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,405 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:38,405 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,405 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,408 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,435 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:38,436 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,436 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,446 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,472 kur.model.executor:597] Finished training on batch. Epoch 1/1, loss=2.249: 29%|██▎ | 2944/10000 [00:00<00:02, 2584.62samples/s][DEBUG 2017-03-01 15:24:38,472 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,479 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,484 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,500 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:38,501 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,501 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,512 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,530 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:38,531 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,539 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,543 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,563 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:38,564 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,564 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,574 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,591 kur.model.executor:597] Finished training on batch. Epoch 1/1, loss=2.228: 35%|██▊ | 3456/10000 [00:01<00:02, 2937.59samples/s][DEBUG 2017-03-01 15:24:38,591 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,601 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,605 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,626 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:38,627 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,627 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,638 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,659 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:38,660 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,660 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,674 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,690 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:38,691 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,701 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,705 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,723 kur.model.executor:597] Finished training on batch. Epoch 1/1, loss=2.210: 40%|███▏ | 3968/10000 [00:01<00:01, 3167.84samples/s][DEBUG 2017-03-01 15:24:38,723 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,732 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,736 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,754 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:38,755 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,765 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,770 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,789 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:38,790 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,790 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,800 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,817 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:38,818 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,818 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,831 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,848 kur.model.executor:597] Finished training on batch. Epoch 1/1, loss=2.188: 45%|███▌ | 4480/10000 [00:01<00:01, 3398.18samples/s][DEBUG 2017-03-01 15:24:38,848 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,856 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,860 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,887 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:38,888 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,888 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,903 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,922 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:38,922 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,923 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,936 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,955 kur.model.executor:597] Finished training on batch. Epoch 1/1, loss=2.178: 49%|███▉ | 4864/10000 [00:01<00:01, 3449.12samples/s][DEBUG 2017-03-01 15:24:38,956 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,956 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:38,970 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:38,990 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:38,990 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:38,998 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,002 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,022 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:39,023 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,023 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,037 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,057 kur.model.executor:597] Finished training on batch. Epoch 1/1, loss=2.167: 52%|████▏ | 5248/10000 [00:01<00:01, 3540.87samples/s][DEBUG 2017-03-01 15:24:39,057 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,058 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,069 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,089 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:39,090 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,090 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,101 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,119 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:39,119 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,130 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,134 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,155 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:39,156 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,156 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,170 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,192 kur.model.executor:597] Finished training on batch. Epoch 1/1, loss=2.152: 58%|████▌ | 5760/10000 [00:01<00:01, 3609.14samples/s][DEBUG 2017-03-01 15:24:39,193 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,203 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,206 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,222 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:39,223 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,223 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,237 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,256 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:39,256 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,263 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,267 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,283 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:39,284 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,284 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,297 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,317 kur.model.executor:597] Finished training on batch. Epoch 1/1, loss=2.135: 63%|█████ | 6272/10000 [00:01<00:00, 3746.78samples/s][DEBUG 2017-03-01 15:24:39,317 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,325 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,328 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,347 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:39,348 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,349 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,358 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,384 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:39,384 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,392 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,396 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,416 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:39,418 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,418 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,427 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,449 kur.model.executor:597] Finished training on batch. Epoch 1/1, loss=2.119: 68%|█████▍ | 6784/10000 [00:01<00:00, 3780.45samples/s][DEBUG 2017-03-01 15:24:39,451 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,451 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,469 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,486 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:39,487 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,487 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,501 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,519 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:39,519 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,527 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,532 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,551 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:39,552 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,552 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,566 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,582 kur.model.executor:597] Finished training on batch. Epoch 1/1, loss=2.104: 73%|█████▊ | 7296/10000 [00:02<00:00, 3803.34samples/s][DEBUG 2017-03-01 15:24:39,583 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,583 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,592 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,615 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:39,616 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,616 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,624 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,646 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:39,647 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,647 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,659 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,678 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:39,679 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,679 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,689 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,712 kur.model.executor:597] Finished training on batch. Epoch 1/1, loss=2.092: 78%|██████▏ | 7808/10000 [00:02<00:00, 3847.20samples/s][DEBUG 2017-03-01 15:24:39,712 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,712 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,724 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,743 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:39,743 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,743 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,754 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,777 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:39,778 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,778 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,779 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,820 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:39,821 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,821 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,833 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,852 kur.model.executor:597] Finished training on batch. Epoch 1/1, loss=2.078: 83%|██████▋ | 8320/10000 [00:02<00:00, 3789.42samples/s][DEBUG 2017-03-01 15:24:39,852 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,852 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,864 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,880 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:39,881 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,881 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,894 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,915 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:39,916 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,916 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,917 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,949 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:39,949 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,950 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:39,965 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:39,987 kur.model.executor:597] Finished training on batch. Epoch 1/1, loss=2.067: 88%|███████ | 8832/10000 [00:02<00:00, 3785.63samples/s][DEBUG 2017-03-01 15:24:39,988 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:39,988 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:40,001 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:40,022 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:40,023 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:40,024 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:40,037 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:40,055 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:40,056 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:40,067 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:40,070 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:40,089 kur.model.executor:597] Finished training on batch. Epoch 1/1, loss=2.058: 92%|███████▎| 9216/10000 [00:02<00:00, 3775.74samples/s][DEBUG 2017-03-01 15:24:40,090 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:40,090 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:40,103 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:40,121 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:40,121 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:40,121 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:40,134 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:40,153 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:40,153 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:40,154 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:40,168 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:40,190 kur.model.executor:597] Finished training on batch. Epoch 1/1, loss=2.052: 96%|███████▋| 9600/10000 [00:02<00:00, 3788.37samples/s][DEBUG 2017-03-01 15:24:40,191 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:40,191 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:40,205 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:40,227 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:40,227 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:40,238 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:40,242 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:40,263 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-01 15:24:40,264 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:40,264 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:40,265 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:40,300 kur.model.executor:597] Finished training on batch. Epoch 1/1, loss=2.043: 100%|███████▉| 9984/10000 [00:02<00:00, 3695.83samples/s][DEBUG 2017-03-01 15:24:40,301 kur.model.executor:578] Training on batch... [DEBUG 2017-03-01 15:24:40,309 kur.model.executor:597] Finished training on batch. Epoch 1/1, loss=2.042: 100%|███████| 10000/10000 [00:02<00:00, 3528.21samples/s] [INFO 2017-03-01 15:24:40,310 kur.model.executor:464] Training loss: 2.042 [INFO 2017-03-01 15:24:40,310 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w [DEBUG 2017-03-01 15:24:40,310 kur.model.executor:430] Saving weights to: t1_dp0.25/cifar.best.train.w [DEBUG 2017-03-01 15:24:40,310 kur.model.model:213] Saving model weights to: t1_dp0.25/cifar.best.train.w [DEBUG 2017-03-01 15:24:40,316 kur.loggers.binary_logger:135] Adding data to binary column: training_loss_total [DEBUG 2017-03-01 15:24:40,317 kur.loggers.binary_logger:135] Adding data to binary column: training_loss_labels [DEBUG 2017-03-01 15:24:40,318 kur.loggers.binary_logger:135] Adding data to binary column: training_loss_batch [DEBUG 2017-03-01 15:24:40,319 kur.loggers.binary_logger:144] Writing logger summary. [DEBUG 2017-03-01 15:24:40,320 kur.loggers.binary_logger:135] Adding data to binary column: batch_loss_total [DEBUG 2017-03-01 15:24:40,321 kur.loggers.binary_logger:135] Adding data to binary column: batch_loss_labels [DEBUG 2017-03-01 15:24:40,322 kur.loggers.binary_logger:135] Adding data to binary column: batch_loss_batch [DEBUG 2017-03-01 15:24:40,323 kur.loggers.binary_logger:144] Writing logger summary. [DEBUG 2017-03-01 15:24:40,325 kur.model.executor:101] Recompiling the model. [DEBUG 2017-03-01 15:24:40,327 kur.backend.keras_backend:543] Reusing an existing model. [DEBUG 2017-03-01 15:24:40,327 kur.backend.keras_backend:560] Assembling a testing function from the model. [DEBUG 2017-03-01 15:24:40,333 kur.backend.keras_backend:509] Adding additional inputs: labels [DEBUG 2017-03-01 15:24:40,707 kur.backend.keras_backend:599] Additional inputs for log functions: labels [DEBUG 2017-03-01 15:24:40,707 kur.backend.keras_backend:616] Expected input shapes: images=(None, 32, 32, 3), labels=(None, None) [DEBUG 2017-03-01 15:24:40,707 kur.backend.keras_backend:634] Compiled model: {'names': {'output': ['labels', 'labels'], 'input': ['images', 'labels']}, 'func': <keras.backend.theano_backend.Function object at 0x10c5e4320>, 'shapes': {'input': [(None, 32, 32, 3), (None, None)]}} [DEBUG 2017-03-01 15:24:40,707 kur.providers.batch_provider:57] Batch size set to: 2 [DEBUG 2017-03-01 15:24:40,708 kur.providers.batch_provider:102] Maximum number of batches set to: 1 [INFO 2017-03-01 15:24:40,713 kur.backend.keras_backend:666] Waiting for model to finish compiling... [DEBUG 2017-03-01 15:24:40,713 kur.providers.batch_provider:139] Preparing next batch of data... [DEBUG 2017-03-01 15:24:40,713 kur.providers.batch_provider:204] Next batch of data has been prepared. Validating, loss=N/A: 0%| | 0/6400 [00:00<?, ?samples/s][DEBUG 2017-03-01 15:24:40,717 kur.providers.shuffle_provider:184] Shuffling... [DEBUG 2017-03-01 15:24:40,853 kur.providers.batch_provider:139] Preparing next batch of data... [DEBUG 2017-03-01 15:24:40,854 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:40,855 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:40,871 kur.providers.batch_provider:204] Next batch of data has been prepared. Validating, loss=1.848: 2%|▏ | 128/6400 [00:00<00:07, 821.70samples/s][DEBUG 2017-03-01 15:24:40,873 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:40,874 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:40,887 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:40,900 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:40,901 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:40,909 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:40,914 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:40,922 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:40,925 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:40,933 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:40,943 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:40,947 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:40,948 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:40,959 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:40,966 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:40,969 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:40,970 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:40,981 kur.providers.batch_provider:204] Next batch of data has been prepared. Validating, loss=1.787: 20%|█▌ | 1280/6400 [00:00<00:04, 1135.88samples/s][DEBUG 2017-03-01 15:24:40,983 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:40,992 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:40,997 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,005 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,009 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,021 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,022 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,030 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,035 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,045 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,055 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,060 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,070 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,075 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,075 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,087 kur.providers.batch_provider:204] Next batch of data has been prepared. Validating, loss=1.799: 36%|██▉ | 2304/6400 [00:00<00:02, 1544.41samples/s][DEBUG 2017-03-01 15:24:41,097 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,102 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,102 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,114 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,119 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,129 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,132 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,145 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,147 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,160 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,160 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,172 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,182 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,188 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,189 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,202 kur.providers.batch_provider:204] Next batch of data has been prepared. Validating, loss=1.809: 52%|████▏ | 3328/6400 [00:00<00:01, 2055.80samples/s][DEBUG 2017-03-01 15:24:41,203 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,213 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,228 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,233 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,244 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,249 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,250 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,256 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,261 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,275 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,283 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,287 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,288 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,297 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,302 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,311 kur.providers.batch_provider:204] Next batch of data has been prepared. Validating, loss=1.811: 68%|█████▍ | 4352/6400 [00:00<00:00, 2677.92samples/s][DEBUG 2017-03-01 15:24:41,323 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,328 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,328 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,337 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,342 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,348 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,353 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,362 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,364 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,373 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,378 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,389 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,391 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,399 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,402 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,412 kur.providers.batch_provider:204] Next batch of data has been prepared. Validating, loss=1.814: 84%|██████▋ | 5376/6400 [00:00<00:00, 3434.82samples/s][DEBUG 2017-03-01 15:24:41,416 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,426 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,431 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,440 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,443 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,453 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,458 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,465 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,469 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,484 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,485 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,492 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-01 15:24:41,497 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-01 15:24:41,504 kur.providers.batch_provider:204] Next batch of data has been prepared. Validating, loss=1.815: 100%|████████| 6400/6400 [00:00<00:00, 4241.72samples/s] [INFO 2017-03-01 15:24:41,525 kur.model.executor:197] Validation loss: 1.815 [INFO 2017-03-01 15:24:41,526 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w [DEBUG 2017-03-01 15:24:41,526 kur.model.executor:444] Copying weights from: t1_dp0.25/cifar.best.train.w [DEBUG 2017-03-01 15:24:41,529 kur.loggers.binary_logger:135] Adding data to binary column: validation_loss_total [DEBUG 2017-03-01 15:24:41,530 kur.loggers.binary_logger:135] Adding data to binary column: validation_loss_labels [DEBUG 2017-03-01 15:24:41,531 kur.loggers.binary_logger:135] Adding data to binary column: validation_loss_batch [DEBUG 2017-03-01 15:24:41,533 kur.loggers.binary_logger:144] Writing logger summary. Completed 1 epochs. [INFO 2017-03-01 15:24:41,537 kur.model.executor:235] Saving most recent weights: t1_dp0.25/cifar.last.w [DEBUG 2017-03-01 15:24:41,537 kur.model.model:213] Saving model weights to: t1_dp0.25/cifar.last.w
%%writefile dlnd_p2_dropout.yml
---
settings:
# Where to get the data
cifar: &cifar
url: "https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz"
checksum: "6d958be074577803d12ecdefd02955f39262c83c16fe9348329d7fe0b5c001ce"
path: "~/kur" # if kur does not have normalization or one-hot-encoding, ##################
# without normalization and one-hot-encoding, the performance will be hurt, right? ####
# Backend to use
backend:
name: keras
# Hyperparameters
# cnn:
# kernels: 20
# size: [5, 5]
# strides: [2, 2]
# pool:
# size: [2,2]
# strides: [2,2]
# type: max
model:
- input: images
- convolution:
kernels: 20
size: [5, 5]
strides: [2,2]
- activation: relu
- pool:
size: [2,2]
strides: [2,2]
type: max
- flatten:
- dense: 15
- dropout: 0.25
- dense: 10
- activation: softmax
name: labels
train:
data:
- cifar:
<<: *cifar
parts: [1]
provider: &provider
batch_size: 128
num_batches: 3 # use all batches
randomize: True
log: t1_dp0.25/cifar-log
epochs: 1
weights:
initial: t1_dp0.25/cifar.best.valid.w
best: t1_dp0.25/cifar.best.train.w
last: t1_dp0.25/cifar.last.w
optimizer:
name: adam
learning_rate: 0.001
validate:
data:
- cifar:
<<: *cifar
parts: 5
provider:
<<: *provider # can I do this? ##########
num_batches: 2 # override num_batches to be 50 while keep batch_size and randomize as 128 and True ####
weights: t1_dp0.25/cifar.best.valid.w
test: &test
data:
- cifar:
<<: *cifar
parts: test
weights: t1_dp0.25/cifar.best.valid.w
provider:
<<: *provider
randomize: False
evaluate:
<<: *test
destination: t1_dp0.25/cifar.results.pkl
loss:
- target: labels # in the project: training loss and valid_accuracy are printed #############
name: categorical_crossentropy # this should be a matter of personal taste, won't really affect anything##
...
Overwriting dlnd_p2_dropout.yml
!kur -vv train dlnd_p2_dropout.yml
[INFO 2017-03-02 20:31:03,897 kur.kurfile:699] Parsing source: dlnd_p2_dropout.yml, included by top-level. [INFO 2017-03-02 20:31:03,910 kur.kurfile:82] Parsing Kurfile... [DEBUG 2017-03-02 20:31:03,910 kur.kurfile:784] Parsing Kurfile section: settings [DEBUG 2017-03-02 20:31:03,914 kur.kurfile:784] Parsing Kurfile section: train [DEBUG 2017-03-02 20:31:03,917 kur.kurfile:784] Parsing Kurfile section: validate [DEBUG 2017-03-02 20:31:03,919 kur.kurfile:784] Parsing Kurfile section: test [DEBUG 2017-03-02 20:31:03,921 kur.kurfile:784] Parsing Kurfile section: evaluate [DEBUG 2017-03-02 20:31:03,925 kur.containers.layers.placeholder:63] Using short-hand name for placeholder: images [DEBUG 2017-03-02 20:31:03,925 kur.containers.layers.placeholder:97] Placeholder "images" has a deferred shape. [DEBUG 2017-03-02 20:31:03,927 kur.kurfile:784] Parsing Kurfile section: loss [INFO 2017-03-02 20:31:03,929 kur.loggers.binary_logger:107] Log does not exist. Creating path: t1_dp0.25/cifar-log [DEBUG 2017-03-02 20:31:04,727 kur.utils.package:233] File exists and passed checksum: /Users/Natsume/kur/cifar-10-python.tar.gz [DEBUG 2017-03-02 20:31:11,034 kur.providers.batch_provider:57] Batch size set to: 128 [DEBUG 2017-03-02 20:31:11,034 kur.providers.batch_provider:102] Maximum number of batches set to: 3 [DEBUG 2017-03-02 20:31:11,766 kur.utils.package:233] File exists and passed checksum: /Users/Natsume/kur/cifar-10-python.tar.gz [DEBUG 2017-03-02 20:31:17,439 kur.providers.batch_provider:57] Batch size set to: 128 [DEBUG 2017-03-02 20:31:17,439 kur.providers.batch_provider:102] Maximum number of batches set to: 2 [DEBUG 2017-03-02 20:31:17,439 kur.backend.backend:187] Using backend: keras [INFO 2017-03-02 20:31:17,440 kur.backend.backend:80] Creating backend: keras [INFO 2017-03-02 20:31:17,440 kur.backend.backend:83] Backend variants: none [INFO 2017-03-02 20:31:17,440 kur.backend.keras_backend:122] No particular backend for Keras has been requested. [DEBUG 2017-03-02 20:31:17,440 kur.backend.keras_backend:124] Using the system-default Keras backend. [DEBUG 2017-03-02 20:31:17,440 kur.backend.keras_backend:189] Overriding environmental variables: {'THEANO_FLAGS': None, 'TF_CPP_MIN_LOG_LEVEL': '1', 'KERAS_BACKEND': None} [INFO 2017-03-02 20:31:18,398 kur.backend.keras_backend:195] Keras is loaded. The backend is: theano [INFO 2017-03-02 20:31:18,399 kur.model.model:260] Enumerating the model containers. [INFO 2017-03-02 20:31:18,399 kur.model.model:265] Assembling the model dependency graph. [DEBUG 2017-03-02 20:31:18,400 kur.model.model:272] Assembled Node: images [DEBUG 2017-03-02 20:31:18,400 kur.model.model:274] Uses: [DEBUG 2017-03-02 20:31:18,400 kur.model.model:276] Used by: ..convolution.0 [DEBUG 2017-03-02 20:31:18,400 kur.model.model:277] Aliases: images [DEBUG 2017-03-02 20:31:18,400 kur.model.model:272] Assembled Node: ..convolution.0 [DEBUG 2017-03-02 20:31:18,400 kur.model.model:274] Uses: images [DEBUG 2017-03-02 20:31:18,400 kur.model.model:276] Used by: ..activation.0 [DEBUG 2017-03-02 20:31:18,400 kur.model.model:277] Aliases: ..convolution.0 [DEBUG 2017-03-02 20:31:18,400 kur.model.model:272] Assembled Node: ..activation.0 [DEBUG 2017-03-02 20:31:18,400 kur.model.model:274] Uses: ..convolution.0 [DEBUG 2017-03-02 20:31:18,400 kur.model.model:276] Used by: ..pool.0 [DEBUG 2017-03-02 20:31:18,401 kur.model.model:277] Aliases: ..activation.0 [DEBUG 2017-03-02 20:31:18,401 kur.model.model:272] Assembled Node: ..pool.0 [DEBUG 2017-03-02 20:31:18,401 kur.model.model:274] Uses: ..activation.0 [DEBUG 2017-03-02 20:31:18,401 kur.model.model:276] Used by: ..flatten.0 [DEBUG 2017-03-02 20:31:18,401 kur.model.model:277] Aliases: ..pool.0 [DEBUG 2017-03-02 20:31:18,401 kur.model.model:272] Assembled Node: ..flatten.0 [DEBUG 2017-03-02 20:31:18,401 kur.model.model:274] Uses: ..pool.0 [DEBUG 2017-03-02 20:31:18,401 kur.model.model:276] Used by: ..dense.0 [DEBUG 2017-03-02 20:31:18,401 kur.model.model:277] Aliases: ..flatten.0 [DEBUG 2017-03-02 20:31:18,401 kur.model.model:272] Assembled Node: ..dense.0 [DEBUG 2017-03-02 20:31:18,401 kur.model.model:274] Uses: ..flatten.0 [DEBUG 2017-03-02 20:31:18,401 kur.model.model:276] Used by: ..dropout.0 [DEBUG 2017-03-02 20:31:18,401 kur.model.model:277] Aliases: ..dense.0 [DEBUG 2017-03-02 20:31:18,401 kur.model.model:272] Assembled Node: ..dropout.0 [DEBUG 2017-03-02 20:31:18,401 kur.model.model:274] Uses: ..dense.0 [DEBUG 2017-03-02 20:31:18,401 kur.model.model:276] Used by: ..dense.1 [DEBUG 2017-03-02 20:31:18,401 kur.model.model:277] Aliases: ..dropout.0 [DEBUG 2017-03-02 20:31:18,401 kur.model.model:272] Assembled Node: ..dense.1 [DEBUG 2017-03-02 20:31:18,401 kur.model.model:274] Uses: ..dropout.0 [DEBUG 2017-03-02 20:31:18,401 kur.model.model:276] Used by: labels [DEBUG 2017-03-02 20:31:18,402 kur.model.model:277] Aliases: ..dense.1 [DEBUG 2017-03-02 20:31:18,402 kur.model.model:272] Assembled Node: labels [DEBUG 2017-03-02 20:31:18,402 kur.model.model:274] Uses: ..dense.1 [DEBUG 2017-03-02 20:31:18,402 kur.model.model:276] Used by: [DEBUG 2017-03-02 20:31:18,402 kur.model.model:277] Aliases: labels [INFO 2017-03-02 20:31:18,402 kur.model.model:280] Connecting the model graph. [DEBUG 2017-03-02 20:31:18,402 kur.model.model:311] Building node: images [DEBUG 2017-03-02 20:31:18,402 kur.model.model:312] Aliases: images [DEBUG 2017-03-02 20:31:18,402 kur.model.model:313] Inputs: [DEBUG 2017-03-02 20:31:18,402 kur.containers.layers.placeholder:117] Creating placeholder for "images" with data type "float32". [DEBUG 2017-03-02 20:31:18,402 kur.model.model:125] Trying to infer shape for input "images" [DEBUG 2017-03-02 20:31:18,402 kur.model.model:143] Inferred shape for input "images": (32, 32, 3) [DEBUG 2017-03-02 20:31:18,402 kur.containers.layers.placeholder:127] Inferred shape: (32, 32, 3) [DEBUG 2017-03-02 20:31:18,408 kur.model.model:382] Value: images [DEBUG 2017-03-02 20:31:18,408 kur.model.model:311] Building node: ..convolution.0 [DEBUG 2017-03-02 20:31:18,408 kur.model.model:312] Aliases: ..convolution.0 [DEBUG 2017-03-02 20:31:18,408 kur.model.model:313] Inputs: [DEBUG 2017-03-02 20:31:18,408 kur.model.model:315] - images: images [DEBUG 2017-03-02 20:31:19,584 kur.model.model:382] Value: Elemwise{add,no_inplace}.0 [DEBUG 2017-03-02 20:31:19,585 kur.model.model:311] Building node: ..activation.0 [DEBUG 2017-03-02 20:31:19,585 kur.model.model:312] Aliases: ..activation.0 [DEBUG 2017-03-02 20:31:19,585 kur.model.model:313] Inputs: [DEBUG 2017-03-02 20:31:19,585 kur.model.model:315] - ..convolution.0: Elemwise{add,no_inplace}.0 [DEBUG 2017-03-02 20:31:19,623 kur.model.model:382] Value: Elemwise{mul,no_inplace}.0 [DEBUG 2017-03-02 20:31:19,623 kur.model.model:311] Building node: ..pool.0 [DEBUG 2017-03-02 20:31:19,623 kur.model.model:312] Aliases: ..pool.0 [DEBUG 2017-03-02 20:31:19,623 kur.model.model:313] Inputs: [DEBUG 2017-03-02 20:31:19,623 kur.model.model:315] - ..activation.0: Elemwise{mul,no_inplace}.0 [DEBUG 2017-03-02 20:31:19,626 kur.model.model:382] Value: DimShuffle{0,2,3,1}.0 [DEBUG 2017-03-02 20:31:19,626 kur.model.model:311] Building node: ..flatten.0 [DEBUG 2017-03-02 20:31:19,626 kur.model.model:312] Aliases: ..flatten.0 [DEBUG 2017-03-02 20:31:19,627 kur.model.model:313] Inputs: [DEBUG 2017-03-02 20:31:19,627 kur.model.model:315] - ..pool.0: DimShuffle{0,2,3,1}.0 [DEBUG 2017-03-02 20:31:19,636 kur.model.model:382] Value: Reshape{2}.0 [DEBUG 2017-03-02 20:31:19,636 kur.model.model:311] Building node: ..dense.0 [DEBUG 2017-03-02 20:31:19,636 kur.model.model:312] Aliases: ..dense.0 [DEBUG 2017-03-02 20:31:19,636 kur.model.model:313] Inputs: [DEBUG 2017-03-02 20:31:19,636 kur.model.model:315] - ..flatten.0: Reshape{2}.0 [DEBUG 2017-03-02 20:31:19,638 kur.model.model:382] Value: Elemwise{add,no_inplace}.0 [DEBUG 2017-03-02 20:31:19,638 kur.model.model:311] Building node: ..dropout.0 [DEBUG 2017-03-02 20:31:19,639 kur.model.model:312] Aliases: ..dropout.0 [DEBUG 2017-03-02 20:31:19,639 kur.model.model:313] Inputs: [DEBUG 2017-03-02 20:31:19,639 kur.model.model:315] - ..dense.0: Elemwise{add,no_inplace}.0 [DEBUG 2017-03-02 20:31:19,752 kur.model.model:382] Value: if{}.0 [DEBUG 2017-03-02 20:31:19,752 kur.model.model:311] Building node: ..dense.1 [DEBUG 2017-03-02 20:31:19,752 kur.model.model:312] Aliases: ..dense.1 [DEBUG 2017-03-02 20:31:19,752 kur.model.model:313] Inputs: [DEBUG 2017-03-02 20:31:19,752 kur.model.model:315] - ..dropout.0: if{}.0 [DEBUG 2017-03-02 20:31:19,753 kur.model.model:382] Value: Elemwise{add,no_inplace}.0 [DEBUG 2017-03-02 20:31:19,753 kur.model.model:311] Building node: labels [DEBUG 2017-03-02 20:31:19,754 kur.model.model:312] Aliases: labels [DEBUG 2017-03-02 20:31:19,754 kur.model.model:313] Inputs: [DEBUG 2017-03-02 20:31:19,754 kur.model.model:315] - ..dense.1: Elemwise{add,no_inplace}.0 [DEBUG 2017-03-02 20:31:19,754 kur.model.model:382] Value: Softmax.0 [INFO 2017-03-02 20:31:19,754 kur.model.model:284] Model inputs: images [INFO 2017-03-02 20:31:19,754 kur.model.model:285] Model outputs: labels [INFO 2017-03-02 20:31:19,754 kur.kurfile:357] Ignoring missing initial weights: t1_dp0.25/cifar.best.valid.w. If this is undesireable, set "must_exist" to "yes" in the approriate "weights" section. [INFO 2017-03-02 20:31:19,754 kur.model.executor:315] No historical training loss available from logs. [INFO 2017-03-02 20:31:19,755 kur.model.executor:323] No historical validation loss available from logs. [INFO 2017-03-02 20:31:19,755 kur.model.executor:329] No previous epochs. [DEBUG 2017-03-02 20:31:19,755 kur.model.executor:353] Epoch handling mode: additional [DEBUG 2017-03-02 20:31:19,755 kur.model.executor:101] Recompiling the model. [DEBUG 2017-03-02 20:31:19,755 kur.backend.keras_backend:527] Instantiating a Keras model. [DEBUG 2017-03-02 20:31:20,003 kur.backend.keras_backend:538] ____________________________________________________________________________________________________ [DEBUG 2017-03-02 20:31:20,003 kur.backend.keras_backend:538] Layer (type) Output Shape Param # Connected to [DEBUG 2017-03-02 20:31:20,003 kur.backend.keras_backend:538] ==================================================================================================== [DEBUG 2017-03-02 20:31:20,003 kur.backend.keras_backend:538] images (InputLayer) (None, 32, 32, 3) 0 [DEBUG 2017-03-02 20:31:20,003 kur.backend.keras_backend:538] ____________________________________________________________________________________________________ [DEBUG 2017-03-02 20:31:20,003 kur.backend.keras_backend:538] ..convolution.0 (Convolution2D) (None, 16, 16, 20) 1520 images[0][0] [DEBUG 2017-03-02 20:31:20,003 kur.backend.keras_backend:538] ____________________________________________________________________________________________________ [DEBUG 2017-03-02 20:31:20,003 kur.backend.keras_backend:538] ..activation.0 (Activation) (None, 16, 16, 20) 0 ..convolution.0[0][0] [DEBUG 2017-03-02 20:31:20,003 kur.backend.keras_backend:538] ____________________________________________________________________________________________________ [DEBUG 2017-03-02 20:31:20,003 kur.backend.keras_backend:538] ..pool.0 (MaxPooling2D) (None, 8, 8, 20) 0 ..activation.0[0][0] [DEBUG 2017-03-02 20:31:20,003 kur.backend.keras_backend:538] ____________________________________________________________________________________________________ [DEBUG 2017-03-02 20:31:20,003 kur.backend.keras_backend:538] ..flatten.0 (Flatten) (None, 1280) 0 ..pool.0[0][0] [DEBUG 2017-03-02 20:31:20,003 kur.backend.keras_backend:538] ____________________________________________________________________________________________________ [DEBUG 2017-03-02 20:31:20,003 kur.backend.keras_backend:538] ..dense.0 (Dense) (None, 15) 19215 ..flatten.0[0][0] [DEBUG 2017-03-02 20:31:20,003 kur.backend.keras_backend:538] ____________________________________________________________________________________________________ [DEBUG 2017-03-02 20:31:20,003 kur.backend.keras_backend:538] ..dropout.0 (Dropout) (None, 15) 0 ..dense.0[0][0] [DEBUG 2017-03-02 20:31:20,003 kur.backend.keras_backend:538] ____________________________________________________________________________________________________ [DEBUG 2017-03-02 20:31:20,003 kur.backend.keras_backend:538] ..dense.1 (Dense) (None, 10) 160 ..dropout.0[0][0] [DEBUG 2017-03-02 20:31:20,003 kur.backend.keras_backend:538] ____________________________________________________________________________________________________ [DEBUG 2017-03-02 20:31:20,004 kur.backend.keras_backend:538] labels (Activation) (None, 10) 0 ..dense.1[0][0] [DEBUG 2017-03-02 20:31:20,004 kur.backend.keras_backend:538] ==================================================================================================== [DEBUG 2017-03-02 20:31:20,004 kur.backend.keras_backend:538] Total params: 20,895 [DEBUG 2017-03-02 20:31:20,004 kur.backend.keras_backend:538] Trainable params: 20,895 [DEBUG 2017-03-02 20:31:20,004 kur.backend.keras_backend:538] Non-trainable params: 0 [DEBUG 2017-03-02 20:31:20,004 kur.backend.keras_backend:538] ____________________________________________________________________________________________________ [DEBUG 2017-03-02 20:31:20,004 kur.backend.keras_backend:538] [DEBUG 2017-03-02 20:31:20,004 kur.backend.keras_backend:576] Assembling a training function from the model. [DEBUG 2017-03-02 20:31:20,011 kur.backend.keras_backend:509] Adding additional inputs: labels [DEBUG 2017-03-02 20:31:21,492 kur.backend.keras_backend:599] Additional inputs for log functions: labels [DEBUG 2017-03-02 20:31:21,492 kur.backend.keras_backend:616] Expected input shapes: images=(None, 32, 32, 3), labels=(None, None) [DEBUG 2017-03-02 20:31:21,492 kur.backend.keras_backend:634] Compiled model: {'names': {'output': ['labels', 'labels'], 'input': ['images', 'labels']}, 'shapes': {'input': [(None, 32, 32, 3), (None, None)]}, 'func': <keras.backend.theano_backend.Function object at 0x11f7def28>} [DEBUG 2017-03-02 20:31:21,493 kur.providers.batch_provider:57] Batch size set to: 2 [DEBUG 2017-03-02 20:31:21,493 kur.providers.batch_provider:102] Maximum number of batches set to: 1 [INFO 2017-03-02 20:31:21,497 kur.backend.keras_backend:666] Waiting for model to finish compiling... [DEBUG 2017-03-02 20:31:21,497 kur.providers.batch_provider:139] Preparing next batch of data... [DEBUG 2017-03-02 20:31:21,497 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-02 20:31:21,551 kur.providers.provider:144] Data source "images": entries=10000, shape=(32, 32, 3) [DEBUG 2017-03-02 20:31:21,551 kur.providers.provider:144] Data source "labels": entries=10000, shape=(10,) Epoch 1/1, loss=N/A: 0%| | 0/384 [00:00<?, ?samples/s][DEBUG 2017-03-02 20:31:21,565 kur.providers.shuffle_provider:184] Shuffling... [DEBUG 2017-03-02 20:31:21,855 kur.providers.batch_provider:139] Preparing next batch of data... [DEBUG 2017-03-02 20:31:21,856 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-02 20:31:21,856 kur.model.executor:578] Training on batch... [DEBUG 2017-03-02 20:31:21,865 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-02 20:31:21,868 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-02 20:31:21,894 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-02 20:31:21,894 kur.loggers.binary_logger:135] Adding data to binary column: batch_loss_batch [DEBUG 2017-03-02 20:31:21,895 kur.loggers.binary_logger:135] Adding data to binary column: batch_loss_total [DEBUG 2017-03-02 20:31:21,895 kur.loggers.binary_logger:135] Adding data to binary column: batch_loss_labels [DEBUG 2017-03-02 20:31:21,896 kur.loggers.binary_logger:144] Writing logger summary. Epoch 1/1, loss=2.316: 33%|████ | 128/384 [00:00<00:00, 384.88samples/s][DEBUG 2017-03-02 20:31:21,898 kur.model.executor:578] Training on batch... [DEBUG 2017-03-02 20:31:21,898 kur.providers.batch_provider:156] Preparing next batch of data... [DEBUG 2017-03-02 20:31:21,911 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-02 20:31:21,936 kur.model.executor:597] Finished training on batch. [DEBUG 2017-03-02 20:31:21,937 kur.model.executor:578] Training on batch... [DEBUG 2017-03-02 20:31:21,964 kur.model.executor:597] Finished training on batch. Epoch 1/1, loss=2.300: 100%|████████████| 384/384 [00:00<00:00, 962.01samples/s] [INFO 2017-03-02 20:31:21,964 kur.model.executor:464] Training loss: 2.300 [INFO 2017-03-02 20:31:21,965 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w [DEBUG 2017-03-02 20:31:21,965 kur.model.executor:430] Saving weights to: t1_dp0.25/cifar.best.train.w [DEBUG 2017-03-02 20:31:21,965 kur.model.model:213] Saving model weights to: t1_dp0.25/cifar.best.train.w [DEBUG 2017-03-02 20:31:21,969 kur.loggers.binary_logger:135] Adding data to binary column: batch_loss_batch [DEBUG 2017-03-02 20:31:21,970 kur.loggers.binary_logger:135] Adding data to binary column: batch_loss_total [DEBUG 2017-03-02 20:31:21,970 kur.loggers.binary_logger:135] Adding data to binary column: batch_loss_labels [DEBUG 2017-03-02 20:31:21,970 kur.loggers.binary_logger:144] Writing logger summary. [DEBUG 2017-03-02 20:31:21,971 kur.loggers.binary_logger:135] Adding data to binary column: training_loss_batch [DEBUG 2017-03-02 20:31:21,972 kur.loggers.binary_logger:135] Adding data to binary column: training_loss_total [DEBUG 2017-03-02 20:31:21,973 kur.loggers.binary_logger:135] Adding data to binary column: training_loss_labels [DEBUG 2017-03-02 20:31:21,973 kur.loggers.binary_logger:144] Writing logger summary. [DEBUG 2017-03-02 20:31:21,974 kur.model.executor:101] Recompiling the model. [DEBUG 2017-03-02 20:31:21,975 kur.backend.keras_backend:543] Reusing an existing model. [DEBUG 2017-03-02 20:31:21,975 kur.backend.keras_backend:560] Assembling a testing function from the model. [DEBUG 2017-03-02 20:31:21,982 kur.backend.keras_backend:509] Adding additional inputs: labels [DEBUG 2017-03-02 20:31:22,340 kur.backend.keras_backend:599] Additional inputs for log functions: labels [DEBUG 2017-03-02 20:31:22,340 kur.backend.keras_backend:616] Expected input shapes: images=(None, 32, 32, 3), labels=(None, None) [DEBUG 2017-03-02 20:31:22,340 kur.backend.keras_backend:634] Compiled model: {'names': {'output': ['labels', 'labels'], 'input': ['images', 'labels']}, 'shapes': {'input': [(None, 32, 32, 3), (None, None)]}, 'func': <keras.backend.theano_backend.Function object at 0x11fc74400>} [DEBUG 2017-03-02 20:31:22,340 kur.providers.batch_provider:57] Batch size set to: 2 [DEBUG 2017-03-02 20:31:22,341 kur.providers.batch_provider:102] Maximum number of batches set to: 1 [INFO 2017-03-02 20:31:22,343 kur.backend.keras_backend:666] Waiting for model to finish compiling... [DEBUG 2017-03-02 20:31:22,343 kur.providers.batch_provider:139] Preparing next batch of data... [DEBUG 2017-03-02 20:31:22,343 kur.providers.batch_provider:204] Next batch of data has been prepared. Validating, loss=N/A: 0%| | 0/256 [00:00<?, ?samples/s][DEBUG 2017-03-02 20:31:22,346 kur.providers.shuffle_provider:184] Shuffling... [DEBUG 2017-03-02 20:31:22,612 kur.providers.batch_provider:139] Preparing next batch of data... [DEBUG 2017-03-02 20:31:22,613 kur.providers.batch_provider:204] Next batch of data has been prepared. [DEBUG 2017-03-02 20:31:22,619 kur.providers.batch_provider:156] Preparing next batch of data... Validating, loss=2.300: 50%|█████▌ | 128/256 [00:00<00:00, 463.55samples/s][DEBUG 2017-03-02 20:31:22,622 kur.providers.batch_provider:204] Next batch of data has been prepared. Validating, loss=2.290: 100%|███████████| 256/256 [00:00<00:00, 900.46samples/s] [INFO 2017-03-02 20:31:22,631 kur.model.executor:197] Validation loss: 2.290 [INFO 2017-03-02 20:31:22,631 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w [DEBUG 2017-03-02 20:31:22,631 kur.model.executor:444] Copying weights from: t1_dp0.25/cifar.best.train.w [DEBUG 2017-03-02 20:31:22,633 kur.loggers.binary_logger:135] Adding data to binary column: validation_loss_batch [DEBUG 2017-03-02 20:31:22,633 kur.loggers.binary_logger:135] Adding data to binary column: validation_loss_total [DEBUG 2017-03-02 20:31:22,634 kur.loggers.binary_logger:135] Adding data to binary column: validation_loss_labels [DEBUG 2017-03-02 20:31:22,634 kur.loggers.binary_logger:144] Writing logger summary. Completed 1 epochs. [INFO 2017-03-02 20:31:22,635 kur.model.executor:235] Saving most recent weights: t1_dp0.25/cifar.last.w [DEBUG 2017-03-02 20:31:22,636 kur.model.model:213] Saving model weights to: t1_dp0.25/cifar.last.w
%%writefile dlnd_p2_dropout.yml
---
settings:
# Where to get the data
cifar: &cifar
url: "https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz"
checksum: "6d958be074577803d12ecdefd02955f39262c83c16fe9348329d7fe0b5c001ce"
path: "~/kur" # if kur does not have normalization or one-hot-encoding, ##################
# without normalization and one-hot-encoding, the performance will be hurt, right? ####
# Backend to use
backend:
name: keras
# Hyperparameters
# cnn:
# kernels: 20
# size: [5, 5]
# strides: [2, 2]
# pool:
# size: [2,2]
# strides: [2,2]
# type: max
model:
- input: images
- convolution:
kernels: 20
size: [5, 5]
strides: [2,2]
- activation: relu
- pool:
size: [2,2]
strides: [2,2]
type: max
- flatten:
- dense: 15
- dropout: 0.25
- dense: 10
- activation: softmax
name: labels
train:
data:
- cifar:
<<: *cifar
parts: [1]
provider: &provider
batch_size: 128
num_batches: 1000 # use all batches
randomize: True
log: t1_dp0.25/cifar-log
epochs: 20
weights:
initial: t1_dp0.25/cifar.best.valid.w
best: t1_dp0.25/cifar.best.train.w
last: t1_dp0.25/cifar.last.w
optimizer:
name: adam
learning_rate: 0.001
# hooks:
# - plot: loss.png
hooks:
- plot:
loss_per_batch: t1_dp0.25/loss1.png
loss_per_time: t1_dp0.25/loss2.png
throughput_per_time: t1_dp0.25/loss3.png
validate:
data:
- cifar:
<<: *cifar
parts: 5
provider:
<<: *provider
num_batches: 50
weights: t1_dp0.25/cifar.best.valid.w
test: &test
data:
- cifar:
<<: *cifar
parts: test
weights: t1_dp0.25/cifar.best.valid.w
provider:
<<: *provider
randomize: False
evaluate:
<<: *test
destination: t1_dp0.25/cifar.results.pkl
loss:
- target: labels
name: categorical_crossentropy
...
Overwriting dlnd_p2_dropout.yml
!kur -v train dlnd_p2_dropout.yml
[INFO 2017-03-02 22:41:19,495 kur.kurfile:699] Parsing source: dlnd_p2_dropout.yml, included by top-level. [INFO 2017-03-02 22:41:19,511 kur.kurfile:82] Parsing Kurfile... [INFO 2017-03-02 22:41:19,532 kur.loggers.binary_logger:107] Log does not exist. Creating path: t1_dp0.25/cifar-log [INFO 2017-03-02 22:41:36,185 kur.backend.backend:80] Creating backend: keras [INFO 2017-03-02 22:41:36,185 kur.backend.backend:83] Backend variants: none [INFO 2017-03-02 22:41:36,185 kur.backend.keras_backend:122] No particular backend for Keras has been requested. [INFO 2017-03-02 22:41:37,112 kur.backend.keras_backend:195] Keras is loaded. The backend is: theano [INFO 2017-03-02 22:41:37,113 kur.model.model:260] Enumerating the model containers. [INFO 2017-03-02 22:41:37,113 kur.model.model:265] Assembling the model dependency graph. [INFO 2017-03-02 22:41:37,113 kur.model.model:280] Connecting the model graph. [INFO 2017-03-02 22:41:38,321 kur.model.model:284] Model inputs: images [INFO 2017-03-02 22:41:38,321 kur.model.model:285] Model outputs: labels [INFO 2017-03-02 22:41:38,321 kur.kurfile:357] Ignoring missing initial weights: t1_dp0.25/cifar.best.valid.w. If this is undesireable, set "must_exist" to "yes" in the approriate "weights" section. [INFO 2017-03-02 22:41:38,322 kur.model.executor:315] No historical training loss available from logs. [INFO 2017-03-02 22:41:38,322 kur.model.executor:323] No historical validation loss available from logs. [INFO 2017-03-02 22:41:38,322 kur.model.executor:329] No previous epochs. [INFO 2017-03-02 22:41:39,765 kur.backend.keras_backend:666] Waiting for model to finish compiling... Epoch 1/20, loss=2.077: 100%|██████| 10000/10000 [00:02<00:00, 3982.49samples/s] [INFO 2017-03-02 22:41:42,548 kur.model.executor:464] Training loss: 2.077 [INFO 2017-03-02 22:41:42,548 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w [INFO 2017-03-02 22:41:42,946 kur.backend.keras_backend:666] Waiting for model to finish compiling... Validating, loss=1.857: 100%|████████| 6400/6400 [00:01<00:00, 5417.04samples/s] [INFO 2017-03-02 22:41:44,131 kur.model.executor:197] Validation loss: 1.857 [INFO 2017-03-02 22:41:44,132 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 2/20, loss=1.788: 100%|██████| 10000/10000 [00:02<00:00, 4656.73samples/s] [INFO 2017-03-02 22:41:47,586 kur.model.executor:464] Training loss: 1.788 [INFO 2017-03-02 22:41:47,586 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.659: 100%|████████| 6400/6400 [00:00<00:00, 9786.11samples/s] [INFO 2017-03-02 22:41:48,252 kur.model.executor:197] Validation loss: 1.659 [INFO 2017-03-02 22:41:48,252 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 3/20, loss=1.646: 100%|██████| 10000/10000 [00:02<00:00, 4510.26samples/s] [INFO 2017-03-02 22:41:51,538 kur.model.executor:464] Training loss: 1.646 [INFO 2017-03-02 22:41:51,538 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.541: 100%|████████| 6400/6400 [00:00<00:00, 9282.69samples/s] [INFO 2017-03-02 22:41:52,243 kur.model.executor:197] Validation loss: 1.541 [INFO 2017-03-02 22:41:52,243 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 4/20, loss=1.557: 100%|██████| 10000/10000 [00:02<00:00, 4589.28samples/s] [INFO 2017-03-02 22:41:55,775 kur.model.executor:464] Training loss: 1.557 [INFO 2017-03-02 22:41:55,775 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.491: 100%|███████| 6400/6400 [00:00<00:00, 10572.41samples/s] [INFO 2017-03-02 22:41:56,389 kur.model.executor:197] Validation loss: 1.491 [INFO 2017-03-02 22:41:56,390 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 5/20, loss=1.500: 100%|██████| 10000/10000 [00:02<00:00, 4751.18samples/s] [INFO 2017-03-02 22:41:59,591 kur.model.executor:464] Training loss: 1.500 [INFO 2017-03-02 22:41:59,591 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.454: 100%|███████| 6400/6400 [00:00<00:00, 10827.84samples/s] [INFO 2017-03-02 22:42:00,192 kur.model.executor:197] Validation loss: 1.454 [INFO 2017-03-02 22:42:00,192 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 6/20, loss=1.462: 100%|██████| 10000/10000 [00:01<00:00, 5120.87samples/s] [INFO 2017-03-02 22:42:03,203 kur.model.executor:464] Training loss: 1.462 [INFO 2017-03-02 22:42:03,203 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.416: 100%|███████| 6400/6400 [00:00<00:00, 11439.95samples/s] [INFO 2017-03-02 22:42:03,774 kur.model.executor:197] Validation loss: 1.416 [INFO 2017-03-02 22:42:03,775 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 7/20, loss=1.420: 100%|██████| 10000/10000 [00:01<00:00, 5101.30samples/s] [INFO 2017-03-02 22:42:06,715 kur.model.executor:464] Training loss: 1.420 [INFO 2017-03-02 22:42:06,715 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.398: 100%|███████| 6400/6400 [00:00<00:00, 11667.38samples/s] [INFO 2017-03-02 22:42:07,271 kur.model.executor:197] Validation loss: 1.398 [INFO 2017-03-02 22:42:07,272 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 8/20, loss=1.404: 100%|██████| 10000/10000 [00:02<00:00, 4778.85samples/s] [INFO 2017-03-02 22:42:10,564 kur.model.executor:464] Training loss: 1.404 [INFO 2017-03-02 22:42:10,565 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.383: 100%|███████| 6400/6400 [00:00<00:00, 10729.32samples/s] [INFO 2017-03-02 22:42:11,171 kur.model.executor:197] Validation loss: 1.383 [INFO 2017-03-02 22:42:11,171 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 9/20, loss=1.358: 100%|██████| 10000/10000 [00:02<00:00, 4984.91samples/s] [INFO 2017-03-02 22:42:14,238 kur.model.executor:464] Training loss: 1.358 [INFO 2017-03-02 22:42:14,238 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.366: 100%|███████| 6400/6400 [00:00<00:00, 11038.42samples/s] [INFO 2017-03-02 22:42:14,827 kur.model.executor:197] Validation loss: 1.366 [INFO 2017-03-02 22:42:14,828 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 10/20, loss=1.342: 100%|█████| 10000/10000 [00:01<00:00, 5038.18samples/s] [INFO 2017-03-02 22:42:17,884 kur.model.executor:464] Training loss: 1.342 [INFO 2017-03-02 22:42:17,885 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.358: 100%|███████| 6400/6400 [00:00<00:00, 10852.61samples/s] [INFO 2017-03-02 22:42:18,485 kur.model.executor:197] Validation loss: 1.358 [INFO 2017-03-02 22:42:18,486 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 11/20, loss=1.315: 100%|█████| 10000/10000 [00:02<00:00, 4786.46samples/s] [INFO 2017-03-02 22:42:21,742 kur.model.executor:464] Training loss: 1.315 [INFO 2017-03-02 22:42:21,742 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.337: 100%|███████| 6400/6400 [00:00<00:00, 11502.12samples/s] [INFO 2017-03-02 22:42:22,308 kur.model.executor:197] Validation loss: 1.337 [INFO 2017-03-02 22:42:22,309 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 12/20, loss=1.285: 100%|█████| 10000/10000 [00:02<00:00, 4919.16samples/s] [INFO 2017-03-02 22:42:25,371 kur.model.executor:464] Training loss: 1.285 [INFO 2017-03-02 22:42:25,371 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.324: 100%|███████| 6400/6400 [00:00<00:00, 11415.50samples/s] [INFO 2017-03-02 22:42:25,940 kur.model.executor:197] Validation loss: 1.324 [INFO 2017-03-02 22:42:25,940 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 13/20, loss=1.269: 100%|█████| 10000/10000 [00:02<00:00, 4059.17samples/s] [INFO 2017-03-02 22:42:29,429 kur.model.executor:464] Training loss: 1.269 [INFO 2017-03-02 22:42:29,429 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.312: 100%|████████| 6400/6400 [00:00<00:00, 7220.12samples/s] [INFO 2017-03-02 22:42:30,326 kur.model.executor:197] Validation loss: 1.312 [INFO 2017-03-02 22:42:30,326 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 14/20, loss=1.243: 100%|█████| 10000/10000 [00:02<00:00, 4731.76samples/s] [INFO 2017-03-02 22:42:33,863 kur.model.executor:464] Training loss: 1.243 [INFO 2017-03-02 22:42:33,863 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.307: 100%|███████| 6400/6400 [00:00<00:00, 10456.27samples/s] [INFO 2017-03-02 22:42:34,483 kur.model.executor:197] Validation loss: 1.307 [INFO 2017-03-02 22:42:34,483 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 15/20, loss=1.240: 100%|█████| 10000/10000 [00:02<00:00, 4682.33samples/s] [INFO 2017-03-02 22:42:37,757 kur.model.executor:464] Training loss: 1.240 [INFO 2017-03-02 22:42:37,757 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.287: 100%|████████| 6400/6400 [00:00<00:00, 9928.56samples/s] [INFO 2017-03-02 22:42:38,409 kur.model.executor:197] Validation loss: 1.287 [INFO 2017-03-02 22:42:38,410 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 16/20, loss=1.205: 100%|█████| 10000/10000 [00:02<00:00, 4778.99samples/s] [INFO 2017-03-02 22:42:41,582 kur.model.executor:464] Training loss: 1.205 [INFO 2017-03-02 22:42:41,582 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.270: 100%|███████| 6400/6400 [00:00<00:00, 11664.23samples/s] [INFO 2017-03-02 22:42:42,139 kur.model.executor:197] Validation loss: 1.270 [INFO 2017-03-02 22:42:42,139 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 17/20, loss=1.193: 100%|█████| 10000/10000 [00:02<00:00, 4769.72samples/s] [INFO 2017-03-02 22:42:45,326 kur.model.executor:464] Training loss: 1.193 [INFO 2017-03-02 22:42:45,326 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.271: 100%|███████| 6400/6400 [00:00<00:00, 10762.94samples/s] [INFO 2017-03-02 22:42:45,928 kur.model.executor:197] Validation loss: 1.271 Epoch 18/20, loss=1.183: 100%|█████| 10000/10000 [00:02<00:00, 3339.14samples/s] [INFO 2017-03-02 22:42:50,414 kur.model.executor:464] Training loss: 1.183 [INFO 2017-03-02 22:42:50,414 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.289: 100%|████████| 6400/6400 [00:00<00:00, 8447.62samples/s] [INFO 2017-03-02 22:42:51,181 kur.model.executor:197] Validation loss: 1.289 Epoch 19/20, loss=1.166: 100%|█████| 10000/10000 [00:02<00:00, 3746.48samples/s] [INFO 2017-03-02 22:42:55,805 kur.model.executor:464] Training loss: 1.166 [INFO 2017-03-02 22:42:55,805 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.258: 100%|████████| 6400/6400 [00:00<00:00, 9802.57samples/s] [INFO 2017-03-02 22:42:56,466 kur.model.executor:197] Validation loss: 1.258 [INFO 2017-03-02 22:42:56,466 kur.model.executor:413] Saving best historical validation weights: t1_dp0.25/cifar.best.valid.w Epoch 20/20, loss=1.142: 100%|█████| 10000/10000 [00:02<00:00, 4070.94samples/s] [INFO 2017-03-02 22:43:00,335 kur.model.executor:464] Training loss: 1.142 [INFO 2017-03-02 22:43:00,335 kur.model.executor:471] Saving best historical training weights: t1_dp0.25/cifar.best.train.w Validating, loss=1.268: 100%|████████| 6400/6400 [00:00<00:00, 9413.33samples/s] [INFO 2017-03-02 22:43:01,026 kur.model.executor:197] Validation loss: 1.268 Completed 20 epochs. [INFO 2017-03-02 22:43:02,257 kur.model.executor:235] Saving most recent weights: t1_dp0.25/cifar.last.w
from IPython.display import Image
Image(width = 500, height=200, retina= True, filename='t1_dp0.25/loss1.png')
Image(width = 500, height=200, retina= True, filename='t1_dp0.25/loss2.png')
Image(width = 500, height=500, retina= True, filename = 't1_dp0.25/loss3.png')
!kur dump dlnd_p2_dropout.yml
{ "evaluate": { "data": [ { "cifar": { "checksum": "6d958be074577803d12ecdefd02955f39262c83c16fe9348329d7fe0b5c001ce", "parts": "test", "path": "~/kur", "url": "https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz" } } ], "destination": "t1_dp0.25/cifar.results.pkl", "provider": { "batch_size": 128, "num_batches": 1000, "randomize": false }, "weights": "t1_dp0.25/cifar.best.valid.w" }, "evaluation": { "data": [ { "cifar": { "checksum": "6d958be074577803d12ecdefd02955f39262c83c16fe9348329d7fe0b5c001ce", "parts": "test", "path": "~/kur", "url": "https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz" } } ], "destination": "t1_dp0.25/cifar.results.pkl", "provider": { "batch_size": 128, "num_batches": 1000, "randomize": false }, "weights": "t1_dp0.25/cifar.best.valid.w" }, "loss": [ { "name": "categorical_crossentropy", "target": "labels" } ], "model": [ { "input": "images" }, { "convolution": { "kernels": 20, "size": [ 5, 5 ], "strides": [ 2, 2 ] } }, { "activation": "relu" }, { "pool": { "size": [ 2, 2 ], "strides": [ 2, 2 ], "type": "max" } }, { "flatten": null }, { "dense": 15 }, { "dropout": 0.25 }, { "dense": 10 }, { "activation": "softmax", "name": "labels" } ], "settings": { "backend": { "name": "keras" }, "cifar": { "checksum": "6d958be074577803d12ecdefd02955f39262c83c16fe9348329d7fe0b5c001ce", "path": "~/kur", "url": "https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz" } }, "test": { "data": [ { "cifar": { "checksum": "6d958be074577803d12ecdefd02955f39262c83c16fe9348329d7fe0b5c001ce", "parts": "test", "path": "~/kur", "url": "https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz" } } ], "provider": { "batch_size": 128, "num_batches": 1000, "randomize": false }, "weights": "t1_dp0.25/cifar.best.valid.w" }, "testing": { "data": [ { "cifar": { "checksum": "6d958be074577803d12ecdefd02955f39262c83c16fe9348329d7fe0b5c001ce", "parts": "test", "path": "~/kur", "url": "https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz" } } ], "provider": { "batch_size": 128, "num_batches": 1000, "randomize": false }, "weights": "t1_dp0.25/cifar.best.valid.w" }, "train": { "data": [ { "cifar": { "checksum": "6d958be074577803d12ecdefd02955f39262c83c16fe9348329d7fe0b5c001ce", "parts": [ 1 ], "path": "~/kur", "url": "https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz" } } ], "epochs": 20, "hooks": [ { "plot": { "loss_per_batch": "t1_dp0.25/loss1.png", "loss_per_time": "t1_dp0.25/loss2.png", "throughput_per_time": "t1_dp0.25/loss3.png" } } ], "log": "t1_dp0.25/cifar-log", "optimizer": { "learning_rate": 0.001, "name": "adam" }, "provider": { "batch_size": 128, "num_batches": 1000, "randomize": true }, "weights": { "best": "t1_dp0.25/cifar.best.train.w", "initial": "t1_dp0.25/cifar.best.valid.w", "last": "t1_dp0.25/cifar.last.w" } }, "training": { "data": [ { "cifar": { "checksum": "6d958be074577803d12ecdefd02955f39262c83c16fe9348329d7fe0b5c001ce", "parts": [ 1 ], "path": "~/kur", "url": "https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz" } } ], "epochs": 20, "hooks": [ { "plot": { "loss_per_batch": "t1_dp0.25/loss1.png", "loss_per_time": "t1_dp0.25/loss2.png", "throughput_per_time": "t1_dp0.25/loss3.png" } } ], "log": "t1_dp0.25/cifar-log", "optimizer": { "learning_rate": 0.001, "name": "adam" }, "provider": { "batch_size": 128, "num_batches": 1000, "randomize": true }, "weights": { "best": "t1_dp0.25/cifar.best.train.w", "initial": "t1_dp0.25/cifar.best.valid.w", "last": "t1_dp0.25/cifar.last.w" } }, "validate": { "data": [ { "cifar": { "checksum": "6d958be074577803d12ecdefd02955f39262c83c16fe9348329d7fe0b5c001ce", "parts": 5, "path": "~/kur", "url": "https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz" } } ], "provider": { "batch_size": 128, "num_batches": 50, "randomize": true }, "weights": "t1_dp0.25/cifar.best.valid.w" }, "validation": { "data": [ { "cifar": { "checksum": "6d958be074577803d12ecdefd02955f39262c83c16fe9348329d7fe0b5c001ce", "parts": 5, "path": "~/kur", "url": "https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz" } } ], "provider": { "batch_size": 128, "num_batches": 50, "randomize": true }, "weights": "t1_dp0.25/cifar.best.valid.w" } }