Python Machine Learning 3rd Edition by Sebastian Raschka & Vahid Mirjalili, Packt Publishing Ltd. 2019
Code Repository: https://github.com/rasbt/python-machine-learning-book-3rd-edition
Code License: MIT License
Note that the optional watermark extension is a small IPython notebook plugin that I developed to make the code reproducible. You can just skip the following line(s).
%load_ext watermark
%watermark -a "Sebastian Raschka & Vahid Mirjalili" -u -d -p numpy,scipy,tensorflow,tensorflow_datasets
Sebastian Raschka & Vahid Mirjalili last updated: 2019-10-29 numpy 1.17.2 scipy 1.3.1 tensorflow 2.0.0 tensorflow_datasets 1.3.0
import tensorflow as tf
import tensorflow_datasets as tfds
import numpy as np
BUFFER_SIZE = 10000
BATCH_SIZE = 64
NUM_EPOCHS = 20
steps_per_epoch = np.ceil(60000 / BATCH_SIZE)
def preprocess(item):
image = item['image']
label = item['label']
image = tf.image.convert_image_dtype(
image, tf.float32)
image = tf.reshape(image, (-1,))
return {'image-pixels':image}, label[..., tf.newaxis]
#Step 1: Defining the input functions (one for training and one for evaluation)
## Step 1: Define the input function for training
def train_input_fn():
datasets = tfds.load(name='mnist')
mnist_train = datasets['train']
dataset = mnist_train.map(preprocess)
dataset = dataset.shuffle(BUFFER_SIZE)
dataset = dataset.batch(BATCH_SIZE)
return dataset.repeat()
## define input-function for evaluation:
def eval_input_fn():
datasets = tfds.load(name='mnist')
mnist_test = datasets['test']
dataset = mnist_test.map(preprocess).batch(BATCH_SIZE)
return dataset
## Step 2: feature column
image_feature_column = tf.feature_column.numeric_column(
key='image-pixels', shape=(28*28))
## Step 3: instantiate the estimator
dnn_classifier = tf.estimator.DNNClassifier(
feature_columns=[image_feature_column],
hidden_units=[32, 16],
n_classes=10,
model_dir='models/mnist-dnn/')
## Step 4: train
dnn_classifier.train(
input_fn=train_input_fn,
steps=NUM_EPOCHS * steps_per_epoch)
INFO:tensorflow:Using default config.
INFO:tensorflow:Using config: {'_model_dir': 'models/mnist-dnn/', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
rewrite_options {
meta_optimizer_iterations: ONE
}
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7fe849bfb860>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
WARNING:tensorflow:From /Users/sebastian/miniconda3/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
WARNING:tensorflow:From /Users/sebastian/miniconda3/lib/python3.7/site-packages/tensorflow_core/python/training/training_util.py:236: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
Downloading and preparing dataset mnist (11.06 MiB) to /Users/sebastian/tensorflow_datasets/mnist/1.0.0...
HBox(children=(IntProgress(value=1, bar_style='info', description='Dl Completed...', max=1, style=ProgressStyl…
HBox(children=(IntProgress(value=1, bar_style='info', description='Dl Size...', max=1, style=ProgressStyle(des…
HBox(children=(IntProgress(value=1, bar_style='info', description='Extraction completed...', max=1, style=Prog…
/Users/sebastian/miniconda3/lib/python3.7/site-packages/urllib3/connectionpool.py:847: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) /Users/sebastian/miniconda3/lib/python3.7/site-packages/urllib3/connectionpool.py:847: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) /Users/sebastian/miniconda3/lib/python3.7/site-packages/urllib3/connectionpool.py:847: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning) /Users/sebastian/miniconda3/lib/python3.7/site-packages/urllib3/connectionpool.py:847: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning)
HBox(children=(IntProgress(value=1, bar_style='info', max=1), HTML(value='')))
HBox(children=(IntProgress(value=0, description='Shuffling...', max=10, style=ProgressStyle(description_width=…
WARNING:tensorflow:From /Users/sebastian/miniconda3/lib/python3.7/site-packages/tensorflow_datasets/core/file_format_adapter.py:209: tf_record_iterator (from tensorflow.python.lib.io.tf_record) is deprecated and will be removed in a future version. Instructions for updating: Use eager execution and: `tf.data.TFRecordDataset(path)`
WARNING:tensorflow:From /Users/sebastian/miniconda3/lib/python3.7/site-packages/tensorflow_datasets/core/file_format_adapter.py:209: tf_record_iterator (from tensorflow.python.lib.io.tf_record) is deprecated and will be removed in a future version. Instructions for updating: Use eager execution and: `tf.data.TFRecordDataset(path)`
HBox(children=(IntProgress(value=1, bar_style='info', description='Reading...', max=1, style=ProgressStyle(des…
HBox(children=(IntProgress(value=0, description='Writing...', max=6000, style=ProgressStyle(description_width=…
HBox(children=(IntProgress(value=1, bar_style='info', description='Reading...', max=1, style=ProgressStyle(des…
HBox(children=(IntProgress(value=0, description='Writing...', max=6000, style=ProgressStyle(description_width=…
HBox(children=(IntProgress(value=1, bar_style='info', description='Reading...', max=1, style=ProgressStyle(des…
HBox(children=(IntProgress(value=0, description='Writing...', max=6000, style=ProgressStyle(description_width=…
HBox(children=(IntProgress(value=1, bar_style='info', description='Reading...', max=1, style=ProgressStyle(des…
HBox(children=(IntProgress(value=0, description='Writing...', max=6000, style=ProgressStyle(description_width=…
HBox(children=(IntProgress(value=1, bar_style='info', description='Reading...', max=1, style=ProgressStyle(des…
HBox(children=(IntProgress(value=0, description='Writing...', max=6000, style=ProgressStyle(description_width=…
HBox(children=(IntProgress(value=1, bar_style='info', description='Reading...', max=1, style=ProgressStyle(des…
HBox(children=(IntProgress(value=0, description='Writing...', max=6000, style=ProgressStyle(description_width=…
HBox(children=(IntProgress(value=1, bar_style='info', description='Reading...', max=1, style=ProgressStyle(des…
HBox(children=(IntProgress(value=0, description='Writing...', max=6000, style=ProgressStyle(description_width=…
HBox(children=(IntProgress(value=1, bar_style='info', description='Reading...', max=1, style=ProgressStyle(des…
HBox(children=(IntProgress(value=0, description='Writing...', max=6000, style=ProgressStyle(description_width=…
HBox(children=(IntProgress(value=1, bar_style='info', description='Reading...', max=1, style=ProgressStyle(des…
HBox(children=(IntProgress(value=0, description='Writing...', max=6000, style=ProgressStyle(description_width=…
HBox(children=(IntProgress(value=1, bar_style='info', description='Reading...', max=1, style=ProgressStyle(des…
HBox(children=(IntProgress(value=0, description='Writing...', max=6000, style=ProgressStyle(description_width=…
HBox(children=(IntProgress(value=1, bar_style='info', max=1), HTML(value='')))
HBox(children=(IntProgress(value=0, description='Shuffling...', max=1, style=ProgressStyle(description_width='…
HBox(children=(IntProgress(value=1, bar_style='info', description='Reading...', max=1, style=ProgressStyle(des…
HBox(children=(IntProgress(value=0, description='Writing...', max=10000, style=ProgressStyle(description_width…
Dataset mnist downloaded and prepared to /Users/sebastian/tensorflow_datasets/mnist/1.0.0. Subsequent calls will reuse this data.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
WARNING:tensorflow:From /Users/sebastian/miniconda3/lib/python3.7/site-packages/tensorflow_core/python/keras/optimizer_v2/adagrad.py:108: calling Constant.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version. Instructions for updating: Call initializer instance with the dtype argument instead of passing it to the constructor
WARNING:tensorflow:From /Users/sebastian/miniconda3/lib/python3.7/site-packages/tensorflow_core/python/keras/optimizer_v2/adagrad.py:108: calling Constant.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version. Instructions for updating: Call initializer instance with the dtype argument instead of passing it to the constructor
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Saving checkpoints for 0 into models/mnist-dnn/model.ckpt.
INFO:tensorflow:Saving checkpoints for 0 into models/mnist-dnn/model.ckpt.
INFO:tensorflow:loss = 2.351057, step = 0
INFO:tensorflow:loss = 2.351057, step = 0
INFO:tensorflow:global_step/sec: 132.942
INFO:tensorflow:global_step/sec: 132.942
INFO:tensorflow:loss = 2.2454062, step = 100 (0.754 sec)
INFO:tensorflow:loss = 2.2454062, step = 100 (0.754 sec)
INFO:tensorflow:global_step/sec: 148.753
INFO:tensorflow:global_step/sec: 148.753
INFO:tensorflow:loss = 2.2050538, step = 200 (0.672 sec)
INFO:tensorflow:loss = 2.2050538, step = 200 (0.672 sec)
INFO:tensorflow:global_step/sec: 156.2
INFO:tensorflow:global_step/sec: 156.2
INFO:tensorflow:loss = 2.0896206, step = 300 (0.640 sec)
INFO:tensorflow:loss = 2.0896206, step = 300 (0.640 sec)
INFO:tensorflow:global_step/sec: 148.775
INFO:tensorflow:global_step/sec: 148.775
INFO:tensorflow:loss = 2.1817794, step = 400 (0.672 sec)
INFO:tensorflow:loss = 2.1817794, step = 400 (0.672 sec)
INFO:tensorflow:global_step/sec: 159.555
INFO:tensorflow:global_step/sec: 159.555
INFO:tensorflow:loss = 2.024548, step = 500 (0.627 sec)
INFO:tensorflow:loss = 2.024548, step = 500 (0.627 sec)
INFO:tensorflow:global_step/sec: 183.997
INFO:tensorflow:global_step/sec: 183.997
INFO:tensorflow:loss = 2.040864, step = 600 (0.544 sec)
INFO:tensorflow:loss = 2.040864, step = 600 (0.544 sec)
INFO:tensorflow:global_step/sec: 185.238
INFO:tensorflow:global_step/sec: 185.238
INFO:tensorflow:loss = 1.9282744, step = 700 (0.540 sec)
INFO:tensorflow:loss = 1.9282744, step = 700 (0.540 sec)
INFO:tensorflow:global_step/sec: 213.271
INFO:tensorflow:global_step/sec: 213.271
INFO:tensorflow:loss = 1.9339888, step = 800 (0.469 sec)
INFO:tensorflow:loss = 1.9339888, step = 800 (0.469 sec)
INFO:tensorflow:global_step/sec: 1179.2
INFO:tensorflow:global_step/sec: 1179.2
INFO:tensorflow:loss = 1.8143462, step = 900 (0.085 sec)
INFO:tensorflow:loss = 1.8143462, step = 900 (0.085 sec)
INFO:tensorflow:global_step/sec: 86.4398
INFO:tensorflow:global_step/sec: 86.4398
INFO:tensorflow:loss = 1.9188311, step = 1000 (1.157 sec)
INFO:tensorflow:loss = 1.9188311, step = 1000 (1.157 sec)
INFO:tensorflow:global_step/sec: 177.215
INFO:tensorflow:global_step/sec: 177.215
INFO:tensorflow:loss = 1.7565705, step = 1100 (0.564 sec)
INFO:tensorflow:loss = 1.7565705, step = 1100 (0.564 sec)
INFO:tensorflow:global_step/sec: 177.87
INFO:tensorflow:global_step/sec: 177.87
INFO:tensorflow:loss = 1.7707877, step = 1200 (0.562 sec)
INFO:tensorflow:loss = 1.7707877, step = 1200 (0.562 sec)
INFO:tensorflow:global_step/sec: 177.579
INFO:tensorflow:global_step/sec: 177.579
INFO:tensorflow:loss = 1.613284, step = 1300 (0.563 sec)
INFO:tensorflow:loss = 1.613284, step = 1300 (0.563 sec)
INFO:tensorflow:global_step/sec: 174.516
INFO:tensorflow:global_step/sec: 174.516
INFO:tensorflow:loss = 1.6403301, step = 1400 (0.573 sec)
INFO:tensorflow:loss = 1.6403301, step = 1400 (0.573 sec)
INFO:tensorflow:global_step/sec: 171.129
INFO:tensorflow:global_step/sec: 171.129
INFO:tensorflow:loss = 1.5022435, step = 1500 (0.584 sec)
INFO:tensorflow:loss = 1.5022435, step = 1500 (0.584 sec)
INFO:tensorflow:global_step/sec: 167.979
INFO:tensorflow:global_step/sec: 167.979
INFO:tensorflow:loss = 1.6234821, step = 1600 (0.596 sec)
INFO:tensorflow:loss = 1.6234821, step = 1600 (0.596 sec)
INFO:tensorflow:global_step/sec: 166.721
INFO:tensorflow:global_step/sec: 166.721
INFO:tensorflow:loss = 1.4866042, step = 1700 (0.600 sec)
INFO:tensorflow:loss = 1.4866042, step = 1700 (0.600 sec)
INFO:tensorflow:global_step/sec: 525.698
INFO:tensorflow:global_step/sec: 525.698
INFO:tensorflow:loss = 1.3214425, step = 1800 (0.190 sec)
INFO:tensorflow:loss = 1.3214425, step = 1800 (0.190 sec)
INFO:tensorflow:global_step/sec: 96.0284
INFO:tensorflow:global_step/sec: 96.0284
INFO:tensorflow:loss = 1.4560758, step = 1900 (1.042 sec)
INFO:tensorflow:loss = 1.4560758, step = 1900 (1.042 sec)
INFO:tensorflow:global_step/sec: 160.745
INFO:tensorflow:global_step/sec: 160.745
INFO:tensorflow:loss = 1.3809863, step = 2000 (0.622 sec)
INFO:tensorflow:loss = 1.3809863, step = 2000 (0.622 sec)
INFO:tensorflow:global_step/sec: 158.711
INFO:tensorflow:global_step/sec: 158.711
INFO:tensorflow:loss = 1.3277512, step = 2100 (0.630 sec)
INFO:tensorflow:loss = 1.3277512, step = 2100 (0.630 sec)
INFO:tensorflow:global_step/sec: 160.204
INFO:tensorflow:global_step/sec: 160.204
INFO:tensorflow:loss = 1.3863884, step = 2200 (0.624 sec)
INFO:tensorflow:loss = 1.3863884, step = 2200 (0.624 sec)
INFO:tensorflow:global_step/sec: 155.356
INFO:tensorflow:global_step/sec: 155.356
INFO:tensorflow:loss = 1.5223844, step = 2300 (0.644 sec)
INFO:tensorflow:loss = 1.5223844, step = 2300 (0.644 sec)
INFO:tensorflow:global_step/sec: 158.033
INFO:tensorflow:global_step/sec: 158.033
INFO:tensorflow:loss = 1.3161225, step = 2400 (0.633 sec)
INFO:tensorflow:loss = 1.3161225, step = 2400 (0.633 sec)
INFO:tensorflow:global_step/sec: 159.532
INFO:tensorflow:global_step/sec: 159.532
INFO:tensorflow:loss = 1.245234, step = 2500 (0.627 sec)
INFO:tensorflow:loss = 1.245234, step = 2500 (0.627 sec)
INFO:tensorflow:global_step/sec: 154.23
INFO:tensorflow:global_step/sec: 154.23
INFO:tensorflow:loss = 1.2514963, step = 2600 (0.649 sec)
INFO:tensorflow:loss = 1.2514963, step = 2600 (0.649 sec)
INFO:tensorflow:global_step/sec: 251.112
INFO:tensorflow:global_step/sec: 251.112
INFO:tensorflow:loss = 1.129127, step = 2700 (0.398 sec)
INFO:tensorflow:loss = 1.129127, step = 2700 (0.398 sec)
INFO:tensorflow:global_step/sec: 1031.47
INFO:tensorflow:global_step/sec: 1031.47
INFO:tensorflow:loss = 1.2411234, step = 2800 (0.097 sec)
INFO:tensorflow:loss = 1.2411234, step = 2800 (0.097 sec)
INFO:tensorflow:global_step/sec: 70.4166
INFO:tensorflow:global_step/sec: 70.4166
INFO:tensorflow:loss = 1.2170045, step = 2900 (1.420 sec)
INFO:tensorflow:loss = 1.2170045, step = 2900 (1.420 sec)
INFO:tensorflow:global_step/sec: 170.408
INFO:tensorflow:global_step/sec: 170.408
INFO:tensorflow:loss = 1.169233, step = 3000 (0.587 sec)
INFO:tensorflow:loss = 1.169233, step = 3000 (0.587 sec)
INFO:tensorflow:global_step/sec: 157.969
INFO:tensorflow:global_step/sec: 157.969
INFO:tensorflow:loss = 1.0771585, step = 3100 (0.633 sec)
INFO:tensorflow:loss = 1.0771585, step = 3100 (0.633 sec)
INFO:tensorflow:global_step/sec: 152.922
INFO:tensorflow:global_step/sec: 152.922
INFO:tensorflow:loss = 1.0976894, step = 3200 (0.654 sec)
INFO:tensorflow:loss = 1.0976894, step = 3200 (0.654 sec)
INFO:tensorflow:global_step/sec: 158.397
INFO:tensorflow:global_step/sec: 158.397
INFO:tensorflow:loss = 1.0426898, step = 3300 (0.631 sec)
INFO:tensorflow:loss = 1.0426898, step = 3300 (0.631 sec)
INFO:tensorflow:global_step/sec: 159.375
INFO:tensorflow:global_step/sec: 159.375
INFO:tensorflow:loss = 1.1440966, step = 3400 (0.627 sec)
INFO:tensorflow:loss = 1.1440966, step = 3400 (0.627 sec)
INFO:tensorflow:global_step/sec: 162.985
INFO:tensorflow:global_step/sec: 162.985
INFO:tensorflow:loss = 1.097571, step = 3500 (0.614 sec)
INFO:tensorflow:loss = 1.097571, step = 3500 (0.614 sec)
INFO:tensorflow:global_step/sec: 167.218
INFO:tensorflow:global_step/sec: 167.218
INFO:tensorflow:loss = 1.111726, step = 3600 (0.598 sec)
INFO:tensorflow:loss = 1.111726, step = 3600 (0.598 sec)
INFO:tensorflow:global_step/sec: 1087.67
INFO:tensorflow:global_step/sec: 1087.67
INFO:tensorflow:loss = 1.020121, step = 3700 (0.092 sec)
INFO:tensorflow:loss = 1.020121, step = 3700 (0.092 sec)
INFO:tensorflow:global_step/sec: 82.1075
INFO:tensorflow:global_step/sec: 82.1075
INFO:tensorflow:loss = 0.9614413, step = 3800 (1.218 sec)
INFO:tensorflow:loss = 0.9614413, step = 3800 (1.218 sec)
INFO:tensorflow:global_step/sec: 157.954
INFO:tensorflow:global_step/sec: 157.954
INFO:tensorflow:loss = 0.9403759, step = 3900 (0.633 sec)
INFO:tensorflow:loss = 0.9403759, step = 3900 (0.633 sec)
INFO:tensorflow:global_step/sec: 158.293
INFO:tensorflow:global_step/sec: 158.293
INFO:tensorflow:loss = 0.95159924, step = 4000 (0.632 sec)
INFO:tensorflow:loss = 0.95159924, step = 4000 (0.632 sec)
INFO:tensorflow:global_step/sec: 156.612
INFO:tensorflow:global_step/sec: 156.612
INFO:tensorflow:loss = 0.9348191, step = 4100 (0.638 sec)
INFO:tensorflow:loss = 0.9348191, step = 4100 (0.638 sec)
INFO:tensorflow:global_step/sec: 157.007
INFO:tensorflow:global_step/sec: 157.007
INFO:tensorflow:loss = 0.9669963, step = 4200 (0.637 sec)
INFO:tensorflow:loss = 0.9669963, step = 4200 (0.637 sec)
INFO:tensorflow:global_step/sec: 156.218
INFO:tensorflow:global_step/sec: 156.218
INFO:tensorflow:loss = 1.0327729, step = 4300 (0.640 sec)
INFO:tensorflow:loss = 1.0327729, step = 4300 (0.640 sec)
INFO:tensorflow:global_step/sec: 155.992
INFO:tensorflow:global_step/sec: 155.992
INFO:tensorflow:loss = 0.90217865, step = 4400 (0.641 sec)
INFO:tensorflow:loss = 0.90217865, step = 4400 (0.641 sec)
INFO:tensorflow:global_step/sec: 155.744
INFO:tensorflow:global_step/sec: 155.744
INFO:tensorflow:loss = 0.81369364, step = 4500 (0.642 sec)
INFO:tensorflow:loss = 0.81369364, step = 4500 (0.642 sec)
INFO:tensorflow:global_step/sec: 368.178
INFO:tensorflow:global_step/sec: 368.178
INFO:tensorflow:loss = 1.0345812, step = 4600 (0.271 sec)
INFO:tensorflow:loss = 1.0345812, step = 4600 (0.271 sec)
INFO:tensorflow:global_step/sec: 96.0097
INFO:tensorflow:global_step/sec: 96.0097
INFO:tensorflow:loss = 0.9349081, step = 4700 (1.042 sec)
INFO:tensorflow:loss = 0.9349081, step = 4700 (1.042 sec)
INFO:tensorflow:global_step/sec: 158.331
INFO:tensorflow:global_step/sec: 158.331
INFO:tensorflow:loss = 0.8972459, step = 4800 (0.632 sec)
INFO:tensorflow:loss = 0.8972459, step = 4800 (0.632 sec)
INFO:tensorflow:global_step/sec: 154.895
INFO:tensorflow:global_step/sec: 154.895
INFO:tensorflow:loss = 1.0068789, step = 4900 (0.646 sec)
INFO:tensorflow:loss = 1.0068789, step = 4900 (0.646 sec)
INFO:tensorflow:global_step/sec: 160.276
INFO:tensorflow:global_step/sec: 160.276
INFO:tensorflow:loss = 0.7107158, step = 5000 (0.624 sec)
INFO:tensorflow:loss = 0.7107158, step = 5000 (0.624 sec)
INFO:tensorflow:global_step/sec: 157.348
INFO:tensorflow:global_step/sec: 157.348
INFO:tensorflow:loss = 0.6492921, step = 5100 (0.635 sec)
INFO:tensorflow:loss = 0.6492921, step = 5100 (0.635 sec)
INFO:tensorflow:global_step/sec: 159.352
INFO:tensorflow:global_step/sec: 159.352
INFO:tensorflow:loss = 1.0066533, step = 5200 (0.628 sec)
INFO:tensorflow:loss = 1.0066533, step = 5200 (0.628 sec)
INFO:tensorflow:global_step/sec: 158.464
INFO:tensorflow:global_step/sec: 158.464
INFO:tensorflow:loss = 0.68421435, step = 5300 (0.631 sec)
INFO:tensorflow:loss = 0.68421435, step = 5300 (0.631 sec)
INFO:tensorflow:global_step/sec: 153.045
INFO:tensorflow:global_step/sec: 153.045
INFO:tensorflow:loss = 0.957071, step = 5400 (0.653 sec)
INFO:tensorflow:loss = 0.957071, step = 5400 (0.653 sec)
INFO:tensorflow:global_step/sec: 205.989
INFO:tensorflow:global_step/sec: 205.989
INFO:tensorflow:loss = 0.81245446, step = 5500 (0.485 sec)
INFO:tensorflow:loss = 0.81245446, step = 5500 (0.485 sec)
INFO:tensorflow:global_step/sec: 1074.89
INFO:tensorflow:global_step/sec: 1074.89
INFO:tensorflow:loss = 0.8264276, step = 5600 (0.093 sec)
INFO:tensorflow:loss = 0.8264276, step = 5600 (0.093 sec)
INFO:tensorflow:global_step/sec: 76.6445
INFO:tensorflow:global_step/sec: 76.6445
INFO:tensorflow:loss = 0.77984166, step = 5700 (1.305 sec)
INFO:tensorflow:loss = 0.77984166, step = 5700 (1.305 sec)
INFO:tensorflow:global_step/sec: 155.062
INFO:tensorflow:global_step/sec: 155.062
INFO:tensorflow:loss = 0.7123084, step = 5800 (0.645 sec)
INFO:tensorflow:loss = 0.7123084, step = 5800 (0.645 sec)
INFO:tensorflow:global_step/sec: 153.489
INFO:tensorflow:global_step/sec: 153.489
INFO:tensorflow:loss = 0.59002125, step = 5900 (0.651 sec)
INFO:tensorflow:loss = 0.59002125, step = 5900 (0.651 sec)
INFO:tensorflow:global_step/sec: 156.283
INFO:tensorflow:global_step/sec: 156.283
INFO:tensorflow:loss = 0.7331785, step = 6000 (0.640 sec)
INFO:tensorflow:loss = 0.7331785, step = 6000 (0.640 sec)
INFO:tensorflow:global_step/sec: 151.671
INFO:tensorflow:global_step/sec: 151.671
INFO:tensorflow:loss = 0.67304885, step = 6100 (0.659 sec)
INFO:tensorflow:loss = 0.67304885, step = 6100 (0.659 sec)
INFO:tensorflow:global_step/sec: 154.576
INFO:tensorflow:global_step/sec: 154.576
INFO:tensorflow:loss = 0.7158637, step = 6200 (0.647 sec)
INFO:tensorflow:loss = 0.7158637, step = 6200 (0.647 sec)
INFO:tensorflow:global_step/sec: 148.425
INFO:tensorflow:global_step/sec: 148.425
INFO:tensorflow:loss = 0.73838997, step = 6300 (0.674 sec)
INFO:tensorflow:loss = 0.73838997, step = 6300 (0.674 sec)
INFO:tensorflow:global_step/sec: 159.122
INFO:tensorflow:global_step/sec: 159.122
INFO:tensorflow:loss = 0.6540515, step = 6400 (0.628 sec)
INFO:tensorflow:loss = 0.6540515, step = 6400 (0.628 sec)
INFO:tensorflow:global_step/sec: 726.68
INFO:tensorflow:global_step/sec: 726.68
INFO:tensorflow:loss = 0.6417879, step = 6500 (0.137 sec)
INFO:tensorflow:loss = 0.6417879, step = 6500 (0.137 sec)
INFO:tensorflow:global_step/sec: 92.173
INFO:tensorflow:global_step/sec: 92.173
INFO:tensorflow:loss = 0.75275004, step = 6600 (1.085 sec)
INFO:tensorflow:loss = 0.75275004, step = 6600 (1.085 sec)
INFO:tensorflow:global_step/sec: 154.247
INFO:tensorflow:global_step/sec: 154.247
INFO:tensorflow:loss = 0.62119484, step = 6700 (0.648 sec)
INFO:tensorflow:loss = 0.62119484, step = 6700 (0.648 sec)
INFO:tensorflow:global_step/sec: 155.557
INFO:tensorflow:global_step/sec: 155.557
INFO:tensorflow:loss = 0.62754023, step = 6800 (0.643 sec)
INFO:tensorflow:loss = 0.62754023, step = 6800 (0.643 sec)
INFO:tensorflow:global_step/sec: 161.037
INFO:tensorflow:global_step/sec: 161.037
INFO:tensorflow:loss = 0.5163309, step = 6900 (0.621 sec)
INFO:tensorflow:loss = 0.5163309, step = 6900 (0.621 sec)
INFO:tensorflow:global_step/sec: 157.524
INFO:tensorflow:global_step/sec: 157.524
INFO:tensorflow:loss = 0.7250298, step = 7000 (0.635 sec)
INFO:tensorflow:loss = 0.7250298, step = 7000 (0.635 sec)
INFO:tensorflow:global_step/sec: 156.113
INFO:tensorflow:global_step/sec: 156.113
INFO:tensorflow:loss = 0.60772896, step = 7100 (0.640 sec)
INFO:tensorflow:loss = 0.60772896, step = 7100 (0.640 sec)
INFO:tensorflow:global_step/sec: 160.343
INFO:tensorflow:global_step/sec: 160.343
INFO:tensorflow:loss = 0.53984725, step = 7200 (0.624 sec)
INFO:tensorflow:loss = 0.53984725, step = 7200 (0.624 sec)
INFO:tensorflow:global_step/sec: 157.021
INFO:tensorflow:global_step/sec: 157.021
INFO:tensorflow:loss = 0.7298105, step = 7300 (0.637 sec)
INFO:tensorflow:loss = 0.7298105, step = 7300 (0.637 sec)
INFO:tensorflow:global_step/sec: 291.906
INFO:tensorflow:global_step/sec: 291.906
INFO:tensorflow:loss = 0.7740065, step = 7400 (0.342 sec)
INFO:tensorflow:loss = 0.7740065, step = 7400 (0.342 sec)
INFO:tensorflow:global_step/sec: 1057.73
INFO:tensorflow:global_step/sec: 1057.73
INFO:tensorflow:loss = 0.72978675, step = 7500 (0.094 sec)
INFO:tensorflow:loss = 0.72978675, step = 7500 (0.094 sec)
INFO:tensorflow:global_step/sec: 69.9693
INFO:tensorflow:global_step/sec: 69.9693
INFO:tensorflow:loss = 0.6730677, step = 7600 (1.429 sec)
INFO:tensorflow:loss = 0.6730677, step = 7600 (1.429 sec)
INFO:tensorflow:global_step/sec: 158.757
INFO:tensorflow:global_step/sec: 158.757
INFO:tensorflow:loss = 0.62808615, step = 7700 (0.630 sec)
INFO:tensorflow:loss = 0.62808615, step = 7700 (0.630 sec)
INFO:tensorflow:global_step/sec: 160.047
INFO:tensorflow:global_step/sec: 160.047
INFO:tensorflow:loss = 0.5663655, step = 7800 (0.625 sec)
INFO:tensorflow:loss = 0.5663655, step = 7800 (0.625 sec)
INFO:tensorflow:global_step/sec: 160.393
INFO:tensorflow:global_step/sec: 160.393
INFO:tensorflow:loss = 0.7629363, step = 7900 (0.624 sec)
INFO:tensorflow:loss = 0.7629363, step = 7900 (0.624 sec)
INFO:tensorflow:global_step/sec: 166.471
INFO:tensorflow:global_step/sec: 166.471
INFO:tensorflow:loss = 0.51022017, step = 8000 (0.601 sec)
INFO:tensorflow:loss = 0.51022017, step = 8000 (0.601 sec)
INFO:tensorflow:global_step/sec: 158.734
INFO:tensorflow:global_step/sec: 158.734
INFO:tensorflow:loss = 0.6464483, step = 8100 (0.630 sec)
INFO:tensorflow:loss = 0.6464483, step = 8100 (0.630 sec)
INFO:tensorflow:global_step/sec: 157.934
INFO:tensorflow:global_step/sec: 157.934
INFO:tensorflow:loss = 0.58257955, step = 8200 (0.633 sec)
INFO:tensorflow:loss = 0.58257955, step = 8200 (0.633 sec)
INFO:tensorflow:global_step/sec: 192.281
INFO:tensorflow:global_step/sec: 192.281
INFO:tensorflow:loss = 0.6757249, step = 8300 (0.520 sec)
INFO:tensorflow:loss = 0.6757249, step = 8300 (0.520 sec)
INFO:tensorflow:global_step/sec: 1070.61
INFO:tensorflow:global_step/sec: 1070.61
INFO:tensorflow:loss = 0.60974616, step = 8400 (0.093 sec)
INFO:tensorflow:loss = 0.60974616, step = 8400 (0.093 sec)
INFO:tensorflow:global_step/sec: 82.3365
INFO:tensorflow:global_step/sec: 82.3365
INFO:tensorflow:loss = 0.5266104, step = 8500 (1.215 sec)
INFO:tensorflow:loss = 0.5266104, step = 8500 (1.215 sec)
INFO:tensorflow:global_step/sec: 161.463
INFO:tensorflow:global_step/sec: 161.463
INFO:tensorflow:loss = 0.57687175, step = 8600 (0.619 sec)
INFO:tensorflow:loss = 0.57687175, step = 8600 (0.619 sec)
INFO:tensorflow:global_step/sec: 168.208
INFO:tensorflow:global_step/sec: 168.208
INFO:tensorflow:loss = 0.5666822, step = 8700 (0.594 sec)
INFO:tensorflow:loss = 0.5666822, step = 8700 (0.594 sec)
INFO:tensorflow:global_step/sec: 167.205
INFO:tensorflow:global_step/sec: 167.205
INFO:tensorflow:loss = 0.55519545, step = 8800 (0.598 sec)
INFO:tensorflow:loss = 0.55519545, step = 8800 (0.598 sec)
INFO:tensorflow:global_step/sec: 167.247
INFO:tensorflow:global_step/sec: 167.247
INFO:tensorflow:loss = 0.5156147, step = 8900 (0.598 sec)
INFO:tensorflow:loss = 0.5156147, step = 8900 (0.598 sec)
INFO:tensorflow:global_step/sec: 168.418
INFO:tensorflow:global_step/sec: 168.418
INFO:tensorflow:loss = 0.6596691, step = 9000 (0.594 sec)
INFO:tensorflow:loss = 0.6596691, step = 9000 (0.594 sec)
INFO:tensorflow:global_step/sec: 169.349
INFO:tensorflow:global_step/sec: 169.349
INFO:tensorflow:loss = 0.5595312, step = 9100 (0.590 sec)
INFO:tensorflow:loss = 0.5595312, step = 9100 (0.590 sec)
INFO:tensorflow:global_step/sec: 166.78
INFO:tensorflow:global_step/sec: 166.78
INFO:tensorflow:loss = 0.40985957, step = 9200 (0.600 sec)
INFO:tensorflow:loss = 0.40985957, step = 9200 (0.600 sec)
INFO:tensorflow:global_step/sec: 475.833
INFO:tensorflow:global_step/sec: 475.833
INFO:tensorflow:loss = 0.53067565, step = 9300 (0.210 sec)
INFO:tensorflow:loss = 0.53067565, step = 9300 (0.210 sec)
INFO:tensorflow:global_step/sec: 99.9111
INFO:tensorflow:global_step/sec: 99.9111
INFO:tensorflow:loss = 0.40128297, step = 9400 (1.001 sec)
INFO:tensorflow:loss = 0.40128297, step = 9400 (1.001 sec)
INFO:tensorflow:global_step/sec: 168.099
INFO:tensorflow:global_step/sec: 168.099
INFO:tensorflow:loss = 0.38595486, step = 9500 (0.595 sec)
INFO:tensorflow:loss = 0.38595486, step = 9500 (0.595 sec)
INFO:tensorflow:global_step/sec: 167.091
INFO:tensorflow:global_step/sec: 167.091
INFO:tensorflow:loss = 0.5257993, step = 9600 (0.599 sec)
INFO:tensorflow:loss = 0.5257993, step = 9600 (0.599 sec)
INFO:tensorflow:global_step/sec: 168.292
INFO:tensorflow:global_step/sec: 168.292
INFO:tensorflow:loss = 0.6027286, step = 9700 (0.594 sec)
INFO:tensorflow:loss = 0.6027286, step = 9700 (0.594 sec)
INFO:tensorflow:global_step/sec: 168.479
INFO:tensorflow:global_step/sec: 168.479
INFO:tensorflow:loss = 0.58924544, step = 9800 (0.594 sec)
INFO:tensorflow:loss = 0.58924544, step = 9800 (0.594 sec)
INFO:tensorflow:global_step/sec: 167.821
INFO:tensorflow:global_step/sec: 167.821
INFO:tensorflow:loss = 0.6070679, step = 9900 (0.596 sec)
INFO:tensorflow:loss = 0.6070679, step = 9900 (0.596 sec)
INFO:tensorflow:global_step/sec: 166.057
INFO:tensorflow:global_step/sec: 166.057
INFO:tensorflow:loss = 0.4609018, step = 10000 (0.602 sec)
INFO:tensorflow:loss = 0.4609018, step = 10000 (0.602 sec)
INFO:tensorflow:global_step/sec: 162.498
INFO:tensorflow:global_step/sec: 162.498
INFO:tensorflow:loss = 0.6651989, step = 10100 (0.615 sec)
INFO:tensorflow:loss = 0.6651989, step = 10100 (0.615 sec)
INFO:tensorflow:global_step/sec: 250.241
INFO:tensorflow:global_step/sec: 250.241
INFO:tensorflow:loss = 0.46471608, step = 10200 (0.399 sec)
INFO:tensorflow:loss = 0.46471608, step = 10200 (0.399 sec)
INFO:tensorflow:global_step/sec: 1117.22
INFO:tensorflow:global_step/sec: 1117.22
INFO:tensorflow:loss = 0.37140846, step = 10300 (0.090 sec)
INFO:tensorflow:loss = 0.37140846, step = 10300 (0.090 sec)
INFO:tensorflow:global_step/sec: 73.522
INFO:tensorflow:global_step/sec: 73.522
INFO:tensorflow:loss = 0.5453825, step = 10400 (1.360 sec)
INFO:tensorflow:loss = 0.5453825, step = 10400 (1.360 sec)
INFO:tensorflow:global_step/sec: 167.571
INFO:tensorflow:global_step/sec: 167.571
INFO:tensorflow:loss = 0.5045433, step = 10500 (0.597 sec)
INFO:tensorflow:loss = 0.5045433, step = 10500 (0.597 sec)
INFO:tensorflow:global_step/sec: 163.145
INFO:tensorflow:global_step/sec: 163.145
INFO:tensorflow:loss = 0.4538327, step = 10600 (0.613 sec)
INFO:tensorflow:loss = 0.4538327, step = 10600 (0.613 sec)
INFO:tensorflow:global_step/sec: 162.434
INFO:tensorflow:global_step/sec: 162.434
INFO:tensorflow:loss = 0.6209466, step = 10700 (0.615 sec)
INFO:tensorflow:loss = 0.6209466, step = 10700 (0.615 sec)
INFO:tensorflow:global_step/sec: 166.714
INFO:tensorflow:global_step/sec: 166.714
INFO:tensorflow:loss = 0.3915943, step = 10800 (0.600 sec)
INFO:tensorflow:loss = 0.3915943, step = 10800 (0.600 sec)
INFO:tensorflow:global_step/sec: 162.328
INFO:tensorflow:global_step/sec: 162.328
INFO:tensorflow:loss = 0.57743776, step = 10900 (0.616 sec)
INFO:tensorflow:loss = 0.57743776, step = 10900 (0.616 sec)
INFO:tensorflow:global_step/sec: 164.609
INFO:tensorflow:global_step/sec: 164.609
INFO:tensorflow:loss = 0.4002396, step = 11000 (0.607 sec)
INFO:tensorflow:loss = 0.4002396, step = 11000 (0.607 sec)
INFO:tensorflow:global_step/sec: 162.761
INFO:tensorflow:global_step/sec: 162.761
INFO:tensorflow:loss = 0.4836612, step = 11100 (0.614 sec)
INFO:tensorflow:loss = 0.4836612, step = 11100 (0.614 sec)
INFO:tensorflow:global_step/sec: 1073.51
INFO:tensorflow:global_step/sec: 1073.51
INFO:tensorflow:loss = 0.5223819, step = 11200 (0.093 sec)
INFO:tensorflow:loss = 0.5223819, step = 11200 (0.093 sec)
INFO:tensorflow:global_step/sec: 89.6572
INFO:tensorflow:global_step/sec: 89.6572
INFO:tensorflow:loss = 0.28695956, step = 11300 (1.115 sec)
INFO:tensorflow:loss = 0.28695956, step = 11300 (1.115 sec)
INFO:tensorflow:global_step/sec: 165.192
INFO:tensorflow:global_step/sec: 165.192
INFO:tensorflow:loss = 0.61684906, step = 11400 (0.605 sec)
INFO:tensorflow:loss = 0.61684906, step = 11400 (0.605 sec)
INFO:tensorflow:global_step/sec: 161.525
INFO:tensorflow:global_step/sec: 161.525
INFO:tensorflow:loss = 0.49000776, step = 11500 (0.619 sec)
INFO:tensorflow:loss = 0.49000776, step = 11500 (0.619 sec)
INFO:tensorflow:global_step/sec: 168.911
INFO:tensorflow:global_step/sec: 168.911
INFO:tensorflow:loss = 0.52533, step = 11600 (0.592 sec)
INFO:tensorflow:loss = 0.52533, step = 11600 (0.592 sec)
INFO:tensorflow:global_step/sec: 161.419
INFO:tensorflow:global_step/sec: 161.419
INFO:tensorflow:loss = 0.5608121, step = 11700 (0.619 sec)
INFO:tensorflow:loss = 0.5608121, step = 11700 (0.619 sec)
INFO:tensorflow:global_step/sec: 167.674
INFO:tensorflow:global_step/sec: 167.674
INFO:tensorflow:loss = 0.48338085, step = 11800 (0.596 sec)
INFO:tensorflow:loss = 0.48338085, step = 11800 (0.596 sec)
INFO:tensorflow:global_step/sec: 162.073
INFO:tensorflow:global_step/sec: 162.073
INFO:tensorflow:loss = 0.3807438, step = 11900 (0.617 sec)
INFO:tensorflow:loss = 0.3807438, step = 11900 (0.617 sec)
INFO:tensorflow:global_step/sec: 165.863
INFO:tensorflow:global_step/sec: 165.863
INFO:tensorflow:loss = 0.5299304, step = 12000 (0.603 sec)
INFO:tensorflow:loss = 0.5299304, step = 12000 (0.603 sec)
INFO:tensorflow:global_step/sec: 346.961
INFO:tensorflow:global_step/sec: 346.961
INFO:tensorflow:loss = 0.41791362, step = 12100 (0.288 sec)
INFO:tensorflow:loss = 0.41791362, step = 12100 (0.288 sec)
INFO:tensorflow:global_step/sec: 108.7
INFO:tensorflow:global_step/sec: 108.7
INFO:tensorflow:loss = 0.45147088, step = 12200 (0.920 sec)
INFO:tensorflow:loss = 0.45147088, step = 12200 (0.920 sec)
INFO:tensorflow:global_step/sec: 161.184
INFO:tensorflow:global_step/sec: 161.184
INFO:tensorflow:loss = 0.5763484, step = 12300 (0.621 sec)
INFO:tensorflow:loss = 0.5763484, step = 12300 (0.621 sec)
INFO:tensorflow:global_step/sec: 163.477
INFO:tensorflow:global_step/sec: 163.477
INFO:tensorflow:loss = 0.4169536, step = 12400 (0.612 sec)
INFO:tensorflow:loss = 0.4169536, step = 12400 (0.612 sec)
INFO:tensorflow:global_step/sec: 164.986
INFO:tensorflow:global_step/sec: 164.986
INFO:tensorflow:loss = 0.4616683, step = 12500 (0.606 sec)
INFO:tensorflow:loss = 0.4616683, step = 12500 (0.606 sec)
INFO:tensorflow:global_step/sec: 161.436
INFO:tensorflow:global_step/sec: 161.436
INFO:tensorflow:loss = 0.5424256, step = 12600 (0.620 sec)
INFO:tensorflow:loss = 0.5424256, step = 12600 (0.620 sec)
INFO:tensorflow:global_step/sec: 158.828
INFO:tensorflow:global_step/sec: 158.828
INFO:tensorflow:loss = 0.36761677, step = 12700 (0.629 sec)
INFO:tensorflow:loss = 0.36761677, step = 12700 (0.629 sec)
INFO:tensorflow:global_step/sec: 167.87
INFO:tensorflow:global_step/sec: 167.87
INFO:tensorflow:loss = 0.34372804, step = 12800 (0.595 sec)
INFO:tensorflow:loss = 0.34372804, step = 12800 (0.595 sec)
INFO:tensorflow:global_step/sec: 163.372
INFO:tensorflow:global_step/sec: 163.372
INFO:tensorflow:loss = 0.42015332, step = 12900 (0.612 sec)
INFO:tensorflow:loss = 0.42015332, step = 12900 (0.612 sec)
INFO:tensorflow:global_step/sec: 209.253
INFO:tensorflow:global_step/sec: 209.253
INFO:tensorflow:loss = 0.48001942, step = 13000 (0.478 sec)
INFO:tensorflow:loss = 0.48001942, step = 13000 (0.478 sec)
INFO:tensorflow:global_step/sec: 1056.94
INFO:tensorflow:global_step/sec: 1056.94
INFO:tensorflow:loss = 0.4390244, step = 13100 (0.095 sec)
INFO:tensorflow:loss = 0.4390244, step = 13100 (0.095 sec)
INFO:tensorflow:global_step/sec: 80.3232
INFO:tensorflow:global_step/sec: 80.3232
INFO:tensorflow:loss = 0.40063435, step = 13200 (1.245 sec)
INFO:tensorflow:loss = 0.40063435, step = 13200 (1.245 sec)
INFO:tensorflow:global_step/sec: 163.099
INFO:tensorflow:global_step/sec: 163.099
INFO:tensorflow:loss = 0.4787082, step = 13300 (0.613 sec)
INFO:tensorflow:loss = 0.4787082, step = 13300 (0.613 sec)
INFO:tensorflow:global_step/sec: 163.875
INFO:tensorflow:global_step/sec: 163.875
INFO:tensorflow:loss = 0.42173663, step = 13400 (0.610 sec)
INFO:tensorflow:loss = 0.42173663, step = 13400 (0.610 sec)
INFO:tensorflow:global_step/sec: 156.275
INFO:tensorflow:global_step/sec: 156.275
INFO:tensorflow:loss = 0.46509022, step = 13500 (0.640 sec)
INFO:tensorflow:loss = 0.46509022, step = 13500 (0.640 sec)
INFO:tensorflow:global_step/sec: 160.013
INFO:tensorflow:global_step/sec: 160.013
INFO:tensorflow:loss = 0.5042197, step = 13600 (0.625 sec)
INFO:tensorflow:loss = 0.5042197, step = 13600 (0.625 sec)
INFO:tensorflow:global_step/sec: 159.285
INFO:tensorflow:global_step/sec: 159.285
INFO:tensorflow:loss = 0.45909923, step = 13700 (0.628 sec)
INFO:tensorflow:loss = 0.45909923, step = 13700 (0.628 sec)
INFO:tensorflow:global_step/sec: 170.196
INFO:tensorflow:global_step/sec: 170.196
INFO:tensorflow:loss = 0.242888, step = 13800 (0.587 sec)
INFO:tensorflow:loss = 0.242888, step = 13800 (0.587 sec)
INFO:tensorflow:global_step/sec: 168.407
INFO:tensorflow:global_step/sec: 168.407
INFO:tensorflow:loss = 0.31859547, step = 13900 (0.594 sec)
INFO:tensorflow:loss = 0.31859547, step = 13900 (0.594 sec)
INFO:tensorflow:global_step/sec: 656.374
INFO:tensorflow:global_step/sec: 656.374
INFO:tensorflow:loss = 0.46284035, step = 14000 (0.152 sec)
INFO:tensorflow:loss = 0.46284035, step = 14000 (0.152 sec)
INFO:tensorflow:global_step/sec: 95.9079
INFO:tensorflow:global_step/sec: 95.9079
INFO:tensorflow:loss = 0.35706022, step = 14100 (1.043 sec)
INFO:tensorflow:loss = 0.35706022, step = 14100 (1.043 sec)
INFO:tensorflow:global_step/sec: 166.735
INFO:tensorflow:global_step/sec: 166.735
INFO:tensorflow:loss = 0.5442808, step = 14200 (0.599 sec)
INFO:tensorflow:loss = 0.5442808, step = 14200 (0.599 sec)
INFO:tensorflow:global_step/sec: 171.925
INFO:tensorflow:global_step/sec: 171.925
INFO:tensorflow:loss = 0.30213034, step = 14300 (0.582 sec)
INFO:tensorflow:loss = 0.30213034, step = 14300 (0.582 sec)
INFO:tensorflow:global_step/sec: 171.289
INFO:tensorflow:global_step/sec: 171.289
INFO:tensorflow:loss = 0.48129684, step = 14400 (0.584 sec)
INFO:tensorflow:loss = 0.48129684, step = 14400 (0.584 sec)
INFO:tensorflow:global_step/sec: 165.133
INFO:tensorflow:global_step/sec: 165.133
INFO:tensorflow:loss = 0.4732051, step = 14500 (0.606 sec)
INFO:tensorflow:loss = 0.4732051, step = 14500 (0.606 sec)
INFO:tensorflow:global_step/sec: 171.167
INFO:tensorflow:global_step/sec: 171.167
INFO:tensorflow:loss = 0.47631317, step = 14600 (0.584 sec)
INFO:tensorflow:loss = 0.47631317, step = 14600 (0.584 sec)
INFO:tensorflow:global_step/sec: 174.224
INFO:tensorflow:global_step/sec: 174.224
INFO:tensorflow:loss = 0.33310536, step = 14700 (0.574 sec)
INFO:tensorflow:loss = 0.33310536, step = 14700 (0.574 sec)
INFO:tensorflow:global_step/sec: 172.043
INFO:tensorflow:global_step/sec: 172.043
INFO:tensorflow:loss = 0.36488658, step = 14800 (0.581 sec)
INFO:tensorflow:loss = 0.36488658, step = 14800 (0.581 sec)
INFO:tensorflow:global_step/sec: 293.538
INFO:tensorflow:global_step/sec: 293.538
INFO:tensorflow:loss = 0.47868153, step = 14900 (0.341 sec)
INFO:tensorflow:loss = 0.47868153, step = 14900 (0.341 sec)
INFO:tensorflow:global_step/sec: 1093.24
INFO:tensorflow:global_step/sec: 1093.24
INFO:tensorflow:loss = 0.34147674, step = 15000 (0.091 sec)
INFO:tensorflow:loss = 0.34147674, step = 15000 (0.091 sec)
INFO:tensorflow:global_step/sec: 76.1293
INFO:tensorflow:global_step/sec: 76.1293
INFO:tensorflow:loss = 0.5065901, step = 15100 (1.314 sec)
INFO:tensorflow:loss = 0.5065901, step = 15100 (1.314 sec)
INFO:tensorflow:global_step/sec: 172.561
INFO:tensorflow:global_step/sec: 172.561
INFO:tensorflow:loss = 0.23874873, step = 15200 (0.580 sec)
INFO:tensorflow:loss = 0.23874873, step = 15200 (0.580 sec)
INFO:tensorflow:global_step/sec: 173.283
INFO:tensorflow:global_step/sec: 173.283
INFO:tensorflow:loss = 0.3538589, step = 15300 (0.577 sec)
INFO:tensorflow:loss = 0.3538589, step = 15300 (0.577 sec)
INFO:tensorflow:global_step/sec: 172.876
INFO:tensorflow:global_step/sec: 172.876
INFO:tensorflow:loss = 0.39499152, step = 15400 (0.578 sec)
INFO:tensorflow:loss = 0.39499152, step = 15400 (0.578 sec)
INFO:tensorflow:global_step/sec: 172.265
INFO:tensorflow:global_step/sec: 172.265
INFO:tensorflow:loss = 0.58030725, step = 15500 (0.580 sec)
INFO:tensorflow:loss = 0.58030725, step = 15500 (0.580 sec)
INFO:tensorflow:global_step/sec: 169.389
INFO:tensorflow:global_step/sec: 169.389
INFO:tensorflow:loss = 0.38975593, step = 15600 (0.590 sec)
INFO:tensorflow:loss = 0.38975593, step = 15600 (0.590 sec)
INFO:tensorflow:global_step/sec: 168.01
INFO:tensorflow:global_step/sec: 168.01
INFO:tensorflow:loss = 0.41811582, step = 15700 (0.595 sec)
INFO:tensorflow:loss = 0.41811582, step = 15700 (0.595 sec)
INFO:tensorflow:global_step/sec: 179.239
INFO:tensorflow:global_step/sec: 179.239
INFO:tensorflow:loss = 0.3553283, step = 15800 (0.558 sec)
INFO:tensorflow:loss = 0.3553283, step = 15800 (0.558 sec)
INFO:tensorflow:global_step/sec: 1045.42
INFO:tensorflow:global_step/sec: 1045.42
INFO:tensorflow:loss = 0.3404338, step = 15900 (0.096 sec)
INFO:tensorflow:loss = 0.3404338, step = 15900 (0.096 sec)
INFO:tensorflow:global_step/sec: 84.389
INFO:tensorflow:global_step/sec: 84.389
INFO:tensorflow:loss = 0.4905302, step = 16000 (1.185 sec)
INFO:tensorflow:loss = 0.4905302, step = 16000 (1.185 sec)
INFO:tensorflow:global_step/sec: 165.86
INFO:tensorflow:global_step/sec: 165.86
INFO:tensorflow:loss = 0.34077603, step = 16100 (0.603 sec)
INFO:tensorflow:loss = 0.34077603, step = 16100 (0.603 sec)
INFO:tensorflow:global_step/sec: 163.717
INFO:tensorflow:global_step/sec: 163.717
INFO:tensorflow:loss = 0.5722649, step = 16200 (0.611 sec)
INFO:tensorflow:loss = 0.5722649, step = 16200 (0.611 sec)
INFO:tensorflow:global_step/sec: 160.81
INFO:tensorflow:global_step/sec: 160.81
INFO:tensorflow:loss = 0.31617087, step = 16300 (0.622 sec)
INFO:tensorflow:loss = 0.31617087, step = 16300 (0.622 sec)
INFO:tensorflow:global_step/sec: 160.882
INFO:tensorflow:global_step/sec: 160.882
INFO:tensorflow:loss = 0.35902232, step = 16400 (0.621 sec)
INFO:tensorflow:loss = 0.35902232, step = 16400 (0.621 sec)
INFO:tensorflow:global_step/sec: 161.608
INFO:tensorflow:global_step/sec: 161.608
INFO:tensorflow:loss = 0.3782361, step = 16500 (0.619 sec)
INFO:tensorflow:loss = 0.3782361, step = 16500 (0.619 sec)
INFO:tensorflow:global_step/sec: 167.816
INFO:tensorflow:global_step/sec: 167.816
INFO:tensorflow:loss = 0.77118593, step = 16600 (0.595 sec)
INFO:tensorflow:loss = 0.77118593, step = 16600 (0.595 sec)
INFO:tensorflow:global_step/sec: 167.034
INFO:tensorflow:global_step/sec: 167.034
INFO:tensorflow:loss = 0.45415357, step = 16700 (0.599 sec)
INFO:tensorflow:loss = 0.45415357, step = 16700 (0.599 sec)
INFO:tensorflow:global_step/sec: 425.006
INFO:tensorflow:global_step/sec: 425.006
INFO:tensorflow:loss = 0.43336082, step = 16800 (0.235 sec)
INFO:tensorflow:loss = 0.43336082, step = 16800 (0.235 sec)
INFO:tensorflow:global_step/sec: 103.148
INFO:tensorflow:global_step/sec: 103.148
INFO:tensorflow:loss = 0.41987512, step = 16900 (0.970 sec)
INFO:tensorflow:loss = 0.41987512, step = 16900 (0.970 sec)
INFO:tensorflow:global_step/sec: 169.634
INFO:tensorflow:global_step/sec: 169.634
INFO:tensorflow:loss = 0.51049364, step = 17000 (0.589 sec)
INFO:tensorflow:loss = 0.51049364, step = 17000 (0.589 sec)
INFO:tensorflow:global_step/sec: 173.82
INFO:tensorflow:global_step/sec: 173.82
INFO:tensorflow:loss = 0.3091932, step = 17100 (0.575 sec)
INFO:tensorflow:loss = 0.3091932, step = 17100 (0.575 sec)
INFO:tensorflow:global_step/sec: 174.611
INFO:tensorflow:global_step/sec: 174.611
INFO:tensorflow:loss = 0.45349842, step = 17200 (0.573 sec)
INFO:tensorflow:loss = 0.45349842, step = 17200 (0.573 sec)
INFO:tensorflow:global_step/sec: 170.322
INFO:tensorflow:global_step/sec: 170.322
INFO:tensorflow:loss = 0.53094894, step = 17300 (0.587 sec)
INFO:tensorflow:loss = 0.53094894, step = 17300 (0.587 sec)
INFO:tensorflow:global_step/sec: 157.377
INFO:tensorflow:global_step/sec: 157.377
INFO:tensorflow:loss = 0.39503157, step = 17400 (0.635 sec)
INFO:tensorflow:loss = 0.39503157, step = 17400 (0.635 sec)
INFO:tensorflow:global_step/sec: 153.652
INFO:tensorflow:global_step/sec: 153.652
INFO:tensorflow:loss = 0.41633907, step = 17500 (0.651 sec)
INFO:tensorflow:loss = 0.41633907, step = 17500 (0.651 sec)
INFO:tensorflow:global_step/sec: 167.355
INFO:tensorflow:global_step/sec: 167.355
INFO:tensorflow:loss = 0.45759898, step = 17600 (0.598 sec)
INFO:tensorflow:loss = 0.45759898, step = 17600 (0.598 sec)
INFO:tensorflow:global_step/sec: 245.045
INFO:tensorflow:global_step/sec: 245.045
INFO:tensorflow:loss = 0.42212462, step = 17700 (0.408 sec)
INFO:tensorflow:loss = 0.42212462, step = 17700 (0.408 sec)
INFO:tensorflow:global_step/sec: 1080.59
INFO:tensorflow:global_step/sec: 1080.59
INFO:tensorflow:loss = 0.4234512, step = 17800 (0.093 sec)
INFO:tensorflow:loss = 0.4234512, step = 17800 (0.093 sec)
INFO:tensorflow:global_step/sec: 73.0558
INFO:tensorflow:global_step/sec: 73.0558
INFO:tensorflow:loss = 0.5996413, step = 17900 (1.371 sec)
INFO:tensorflow:loss = 0.5996413, step = 17900 (1.371 sec)
INFO:tensorflow:global_step/sec: 136.49
INFO:tensorflow:global_step/sec: 136.49
INFO:tensorflow:loss = 0.42301935, step = 18000 (0.731 sec)
INFO:tensorflow:loss = 0.42301935, step = 18000 (0.731 sec)
INFO:tensorflow:global_step/sec: 145.236
INFO:tensorflow:global_step/sec: 145.236
INFO:tensorflow:loss = 0.50754887, step = 18100 (0.688 sec)
INFO:tensorflow:loss = 0.50754887, step = 18100 (0.688 sec)
INFO:tensorflow:global_step/sec: 165.009
INFO:tensorflow:global_step/sec: 165.009
INFO:tensorflow:loss = 0.28448996, step = 18200 (0.606 sec)
INFO:tensorflow:loss = 0.28448996, step = 18200 (0.606 sec)
INFO:tensorflow:global_step/sec: 165.689
INFO:tensorflow:global_step/sec: 165.689
INFO:tensorflow:loss = 0.33631694, step = 18300 (0.604 sec)
INFO:tensorflow:loss = 0.33631694, step = 18300 (0.604 sec)
INFO:tensorflow:global_step/sec: 165.404
INFO:tensorflow:global_step/sec: 165.404
INFO:tensorflow:loss = 0.4742888, step = 18400 (0.605 sec)
INFO:tensorflow:loss = 0.4742888, step = 18400 (0.605 sec)
INFO:tensorflow:global_step/sec: 157.91
INFO:tensorflow:global_step/sec: 157.91
INFO:tensorflow:loss = 0.4124382, step = 18500 (0.633 sec)
INFO:tensorflow:loss = 0.4124382, step = 18500 (0.633 sec)
INFO:tensorflow:global_step/sec: 161.492
INFO:tensorflow:global_step/sec: 161.492
INFO:tensorflow:loss = 0.25739223, step = 18600 (0.620 sec)
INFO:tensorflow:loss = 0.25739223, step = 18600 (0.620 sec)
INFO:tensorflow:global_step/sec: 983.904
INFO:tensorflow:global_step/sec: 983.904
INFO:tensorflow:loss = 0.42343718, step = 18700 (0.101 sec)
INFO:tensorflow:loss = 0.42343718, step = 18700 (0.101 sec)
INFO:tensorflow:Saving checkpoints for 18760 into models/mnist-dnn/model.ckpt.
INFO:tensorflow:Saving checkpoints for 18760 into models/mnist-dnn/model.ckpt.
INFO:tensorflow:Loss for final step: 0.22432359.
INFO:tensorflow:Loss for final step: 0.22432359.
<tensorflow_estimator.python.estimator.canned.dnn.DNNClassifierV2 at 0x7fe849be4e10>
eval_result = dnn_classifier.evaluate(
input_fn=eval_input_fn)
print(eval_result)
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2019-10-29T21:22:53Z
INFO:tensorflow:Starting evaluation at 2019-10-29T21:22:53Z
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from models/mnist-dnn/model.ckpt-18760
INFO:tensorflow:Restoring parameters from models/mnist-dnn/model.ckpt-18760
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Finished evaluation at 2019-10-29-21:22:55
INFO:tensorflow:Finished evaluation at 2019-10-29-21:22:55
INFO:tensorflow:Saving dict for global step 18760: accuracy = 0.8977, average_loss = 0.39452052, global_step = 18760, loss = 0.39615202
INFO:tensorflow:Saving dict for global step 18760: accuracy = 0.8977, average_loss = 0.39452052, global_step = 18760, loss = 0.39615202
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 18760: models/mnist-dnn/model.ckpt-18760
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 18760: models/mnist-dnn/model.ckpt-18760
{'accuracy': 0.8977, 'average_loss': 0.39452052, 'loss': 0.39615202, 'global_step': 18760}
## Set random seeds for reproducibility
tf.random.set_seed(1)
np.random.seed(1)
## Create the data
x = np.random.uniform(low=-1, high=1, size=(200, 2))
y = np.ones(len(x))
y[x[:, 0] * x[:, 1]<0] = 0
x_train = x[:100, :]
y_train = y[:100]
x_valid = x[100:, :]
y_valid = y[100:]
## Step 1: Define the input functions
def train_input_fn(x_train, y_train, batch_size=8):
dataset = tf.data.Dataset.from_tensor_slices(
({'input-features':x_train}, y_train.reshape(-1, 1)))
# Shuffle, repeat, and batch the examples.
return dataset.shuffle(100).repeat().batch(batch_size)
def eval_input_fn(x_test, y_test=None, batch_size=8):
if y_test is None:
dataset = tf.data.Dataset.from_tensor_slices(
{'input-features':x_test})
else:
dataset = tf.data.Dataset.from_tensor_slices(
({'input-features':x_test}, y_test.reshape(-1, 1)))
# Shuffle, repeat, and batch the examples.
return dataset.batch(batch_size)
## Step 2: Define the feature columns
features = [
tf.feature_column.numeric_column(
key='input-features:', shape=(2,))
]
features
[NumericColumn(key='input-features:', shape=(2,), default_value=None, dtype=tf.float32, normalizer_fn=None)]
## Step 3: Create the estimator: convert from a Keras model
model = tf.keras.Sequential([
tf.keras.layers.Input(shape=(2,), name='input-features'),
tf.keras.layers.Dense(units=4, activation='relu'),
tf.keras.layers.Dense(units=4, activation='relu'),
tf.keras.layers.Dense(units=4, activation='relu'),
tf.keras.layers.Dense(1, activation='sigmoid')
])
model.summary()
model.compile(optimizer=tf.keras.optimizers.SGD(),
loss=tf.keras.losses.BinaryCrossentropy(),
metrics=[tf.keras.metrics.BinaryAccuracy()])
my_estimator = tf.keras.estimator.model_to_estimator(
keras_model=model,
model_dir='models/estimator-for-XOR/')
Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= dense (Dense) (None, 4) 12 _________________________________________________________________ dense_1 (Dense) (None, 4) 20 _________________________________________________________________ dense_2 (Dense) (None, 4) 20 _________________________________________________________________ dense_3 (Dense) (None, 1) 5 ================================================================= Total params: 57 Trainable params: 57 Non-trainable params: 0 _________________________________________________________________ INFO:tensorflow:Using default config.
INFO:tensorflow:Using default config.
INFO:tensorflow:Using the Keras model provided.
INFO:tensorflow:Using the Keras model provided.
INFO:tensorflow:Using config: {'_model_dir': 'models/estimator-for-XOR/', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true graph_options { rewrite_options { meta_optimizer_iterations: ONE } } , '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7fe839329780>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
INFO:tensorflow:Using config: {'_model_dir': 'models/estimator-for-XOR/', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true graph_options { rewrite_options { meta_optimizer_iterations: ONE } } , '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7fe839329780>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
## Step 4: use the estimator: train/evaluate/predict
num_epochs = 200
batch_size = 2
steps_per_epoch = np.ceil(len(x_train) / batch_size)
my_estimator.train(
input_fn=lambda: train_input_fn(x_train, y_train, batch_size),
steps=num_epochs * steps_per_epoch)
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Warm-starting with WarmStartSettings: WarmStartSettings(ckpt_to_initialize_from='models/estimator-for-XOR/keras/keras_model.ckpt', vars_to_warm_start='.*', var_name_to_vocab_info={}, var_name_to_prev_var_name={})
INFO:tensorflow:Warm-starting with WarmStartSettings: WarmStartSettings(ckpt_to_initialize_from='models/estimator-for-XOR/keras/keras_model.ckpt', vars_to_warm_start='.*', var_name_to_vocab_info={}, var_name_to_prev_var_name={})
INFO:tensorflow:Warm-starting from: models/estimator-for-XOR/keras/keras_model.ckpt
INFO:tensorflow:Warm-starting from: models/estimator-for-XOR/keras/keras_model.ckpt
INFO:tensorflow:Warm-starting variables only in TRAINABLE_VARIABLES.
INFO:tensorflow:Warm-starting variables only in TRAINABLE_VARIABLES.
INFO:tensorflow:Warm-started 8 variables.
INFO:tensorflow:Warm-started 8 variables.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Saving checkpoints for 0 into models/estimator-for-XOR/model.ckpt.
INFO:tensorflow:Saving checkpoints for 0 into models/estimator-for-XOR/model.ckpt.
INFO:tensorflow:loss = 0.67599964, step = 0
INFO:tensorflow:loss = 0.67599964, step = 0
INFO:tensorflow:global_step/sec: 1031.16
INFO:tensorflow:global_step/sec: 1031.16
INFO:tensorflow:loss = 0.66389585, step = 100 (0.098 sec)
INFO:tensorflow:loss = 0.66389585, step = 100 (0.098 sec)
INFO:tensorflow:global_step/sec: 2229.1
INFO:tensorflow:global_step/sec: 2229.1
INFO:tensorflow:loss = 0.69760054, step = 200 (0.045 sec)
INFO:tensorflow:loss = 0.69760054, step = 200 (0.045 sec)
INFO:tensorflow:global_step/sec: 2045.16
INFO:tensorflow:global_step/sec: 2045.16
INFO:tensorflow:loss = 0.64448076, step = 300 (0.049 sec)
INFO:tensorflow:loss = 0.64448076, step = 300 (0.049 sec)
INFO:tensorflow:global_step/sec: 2124.04
INFO:tensorflow:global_step/sec: 2124.04
INFO:tensorflow:loss = 0.7329457, step = 400 (0.047 sec)
INFO:tensorflow:loss = 0.7329457, step = 400 (0.047 sec)
INFO:tensorflow:global_step/sec: 2099.51
INFO:tensorflow:global_step/sec: 2099.51
INFO:tensorflow:loss = 0.735602, step = 500 (0.047 sec)
INFO:tensorflow:loss = 0.735602, step = 500 (0.047 sec)
INFO:tensorflow:global_step/sec: 2092.14
INFO:tensorflow:global_step/sec: 2092.14
INFO:tensorflow:loss = 0.6452929, step = 600 (0.048 sec)
INFO:tensorflow:loss = 0.6452929, step = 600 (0.048 sec)
INFO:tensorflow:global_step/sec: 2012.06
INFO:tensorflow:global_step/sec: 2012.06
INFO:tensorflow:loss = 0.6274626, step = 700 (0.050 sec)
INFO:tensorflow:loss = 0.6274626, step = 700 (0.050 sec)
INFO:tensorflow:global_step/sec: 2081.3
INFO:tensorflow:global_step/sec: 2081.3
INFO:tensorflow:loss = 0.68258405, step = 800 (0.048 sec)
INFO:tensorflow:loss = 0.68258405, step = 800 (0.048 sec)
INFO:tensorflow:global_step/sec: 2060.75
INFO:tensorflow:global_step/sec: 2060.75
INFO:tensorflow:loss = 0.6896081, step = 900 (0.049 sec)
INFO:tensorflow:loss = 0.6896081, step = 900 (0.049 sec)
INFO:tensorflow:global_step/sec: 2140.55
INFO:tensorflow:global_step/sec: 2140.55
INFO:tensorflow:loss = 0.6942914, step = 1000 (0.047 sec)
INFO:tensorflow:loss = 0.6942914, step = 1000 (0.047 sec)
INFO:tensorflow:global_step/sec: 2111.04
INFO:tensorflow:global_step/sec: 2111.04
INFO:tensorflow:loss = 0.6800655, step = 1100 (0.047 sec)
INFO:tensorflow:loss = 0.6800655, step = 1100 (0.047 sec)
INFO:tensorflow:global_step/sec: 1856.39
INFO:tensorflow:global_step/sec: 1856.39
INFO:tensorflow:loss = 0.70231795, step = 1200 (0.054 sec)
INFO:tensorflow:loss = 0.70231795, step = 1200 (0.054 sec)
INFO:tensorflow:global_step/sec: 1910.4
INFO:tensorflow:global_step/sec: 1910.4
INFO:tensorflow:loss = 0.7397743, step = 1300 (0.052 sec)
INFO:tensorflow:loss = 0.7397743, step = 1300 (0.052 sec)
INFO:tensorflow:global_step/sec: 2009.45
INFO:tensorflow:global_step/sec: 2009.45
INFO:tensorflow:loss = 0.69408834, step = 1400 (0.050 sec)
INFO:tensorflow:loss = 0.69408834, step = 1400 (0.050 sec)
INFO:tensorflow:global_step/sec: 1985.23
INFO:tensorflow:global_step/sec: 1985.23
INFO:tensorflow:loss = 0.7528011, step = 1500 (0.050 sec)
INFO:tensorflow:loss = 0.7528011, step = 1500 (0.050 sec)
INFO:tensorflow:global_step/sec: 2006.86
INFO:tensorflow:global_step/sec: 2006.86
INFO:tensorflow:loss = 0.6945518, step = 1600 (0.050 sec)
INFO:tensorflow:loss = 0.6945518, step = 1600 (0.050 sec)
INFO:tensorflow:global_step/sec: 1909.74
INFO:tensorflow:global_step/sec: 1909.74
INFO:tensorflow:loss = 0.6146264, step = 1700 (0.052 sec)
INFO:tensorflow:loss = 0.6146264, step = 1700 (0.052 sec)
INFO:tensorflow:global_step/sec: 2079.31
INFO:tensorflow:global_step/sec: 2079.31
INFO:tensorflow:loss = 0.65715224, step = 1800 (0.048 sec)
INFO:tensorflow:loss = 0.65715224, step = 1800 (0.048 sec)
INFO:tensorflow:global_step/sec: 2007.72
INFO:tensorflow:global_step/sec: 2007.72
INFO:tensorflow:loss = 0.64627206, step = 1900 (0.050 sec)
INFO:tensorflow:loss = 0.64627206, step = 1900 (0.050 sec)
INFO:tensorflow:global_step/sec: 1950.83
INFO:tensorflow:global_step/sec: 1950.83
INFO:tensorflow:loss = 0.7336613, step = 2000 (0.051 sec)
INFO:tensorflow:loss = 0.7336613, step = 2000 (0.051 sec)
INFO:tensorflow:global_step/sec: 1948.98
INFO:tensorflow:global_step/sec: 1948.98
INFO:tensorflow:loss = 0.69352937, step = 2100 (0.051 sec)
INFO:tensorflow:loss = 0.69352937, step = 2100 (0.051 sec)
INFO:tensorflow:global_step/sec: 1687.34
INFO:tensorflow:global_step/sec: 1687.34
INFO:tensorflow:loss = 0.7212291, step = 2200 (0.059 sec)
INFO:tensorflow:loss = 0.7212291, step = 2200 (0.059 sec)
INFO:tensorflow:global_step/sec: 1898.24
INFO:tensorflow:global_step/sec: 1898.24
INFO:tensorflow:loss = 0.6852537, step = 2300 (0.053 sec)
INFO:tensorflow:loss = 0.6852537, step = 2300 (0.053 sec)
INFO:tensorflow:global_step/sec: 2019.88
INFO:tensorflow:global_step/sec: 2019.88
INFO:tensorflow:loss = 0.54438925, step = 2400 (0.049 sec)
INFO:tensorflow:loss = 0.54438925, step = 2400 (0.049 sec)
INFO:tensorflow:global_step/sec: 1935.91
INFO:tensorflow:global_step/sec: 1935.91
INFO:tensorflow:loss = 0.55820775, step = 2500 (0.052 sec)
INFO:tensorflow:loss = 0.55820775, step = 2500 (0.052 sec)
INFO:tensorflow:global_step/sec: 1992.99
INFO:tensorflow:global_step/sec: 1992.99
INFO:tensorflow:loss = 0.66507506, step = 2600 (0.050 sec)
INFO:tensorflow:loss = 0.66507506, step = 2600 (0.050 sec)
INFO:tensorflow:global_step/sec: 2003.13
INFO:tensorflow:global_step/sec: 2003.13
INFO:tensorflow:loss = 0.43656868, step = 2700 (0.050 sec)
INFO:tensorflow:loss = 0.43656868, step = 2700 (0.050 sec)
INFO:tensorflow:global_step/sec: 1979.96
INFO:tensorflow:global_step/sec: 1979.96
INFO:tensorflow:loss = 0.63751894, step = 2800 (0.051 sec)
INFO:tensorflow:loss = 0.63751894, step = 2800 (0.051 sec)
INFO:tensorflow:global_step/sec: 1932.48
INFO:tensorflow:global_step/sec: 1932.48
INFO:tensorflow:loss = 0.45888674, step = 2900 (0.052 sec)
INFO:tensorflow:loss = 0.45888674, step = 2900 (0.052 sec)
INFO:tensorflow:global_step/sec: 2005.83
INFO:tensorflow:global_step/sec: 2005.83
INFO:tensorflow:loss = 0.68522173, step = 3000 (0.050 sec)
INFO:tensorflow:loss = 0.68522173, step = 3000 (0.050 sec)
INFO:tensorflow:global_step/sec: 2057.6
INFO:tensorflow:global_step/sec: 2057.6
INFO:tensorflow:loss = 0.5820494, step = 3100 (0.048 sec)
INFO:tensorflow:loss = 0.5820494, step = 3100 (0.048 sec)
INFO:tensorflow:global_step/sec: 2030.71
INFO:tensorflow:global_step/sec: 2030.71
INFO:tensorflow:loss = 0.1965836, step = 3200 (0.049 sec)
INFO:tensorflow:loss = 0.1965836, step = 3200 (0.049 sec)
INFO:tensorflow:global_step/sec: 2016.58
INFO:tensorflow:global_step/sec: 2016.58
INFO:tensorflow:loss = 0.48559472, step = 3300 (0.050 sec)
INFO:tensorflow:loss = 0.48559472, step = 3300 (0.050 sec)
INFO:tensorflow:global_step/sec: 2044.57
INFO:tensorflow:global_step/sec: 2044.57
INFO:tensorflow:loss = 0.370632, step = 3400 (0.049 sec)
INFO:tensorflow:loss = 0.370632, step = 3400 (0.049 sec)
INFO:tensorflow:global_step/sec: 2020.45
INFO:tensorflow:global_step/sec: 2020.45
INFO:tensorflow:loss = 0.74945277, step = 3500 (0.050 sec)
INFO:tensorflow:loss = 0.74945277, step = 3500 (0.050 sec)
INFO:tensorflow:global_step/sec: 2114.97
INFO:tensorflow:global_step/sec: 2114.97
INFO:tensorflow:loss = 0.74562764, step = 3600 (0.047 sec)
INFO:tensorflow:loss = 0.74562764, step = 3600 (0.047 sec)
INFO:tensorflow:global_step/sec: 2075.85
INFO:tensorflow:global_step/sec: 2075.85
INFO:tensorflow:loss = 0.41700324, step = 3700 (0.048 sec)
INFO:tensorflow:loss = 0.41700324, step = 3700 (0.048 sec)
INFO:tensorflow:global_step/sec: 2187.13
INFO:tensorflow:global_step/sec: 2187.13
INFO:tensorflow:loss = 0.3938596, step = 3800 (0.046 sec)
INFO:tensorflow:loss = 0.3938596, step = 3800 (0.046 sec)
INFO:tensorflow:global_step/sec: 2150.86
INFO:tensorflow:global_step/sec: 2150.86
INFO:tensorflow:loss = 0.4955755, step = 3900 (0.046 sec)
INFO:tensorflow:loss = 0.4955755, step = 3900 (0.046 sec)
INFO:tensorflow:global_step/sec: 2163.47
INFO:tensorflow:global_step/sec: 2163.47
INFO:tensorflow:loss = 0.51575875, step = 4000 (0.046 sec)
INFO:tensorflow:loss = 0.51575875, step = 4000 (0.046 sec)
INFO:tensorflow:global_step/sec: 2145.37
INFO:tensorflow:global_step/sec: 2145.37
INFO:tensorflow:loss = 0.5289919, step = 4100 (0.047 sec)
INFO:tensorflow:loss = 0.5289919, step = 4100 (0.047 sec)
INFO:tensorflow:global_step/sec: 2040.49
INFO:tensorflow:global_step/sec: 2040.49
INFO:tensorflow:loss = 0.3431827, step = 4200 (0.049 sec)
INFO:tensorflow:loss = 0.3431827, step = 4200 (0.049 sec)
INFO:tensorflow:global_step/sec: 1919.23
INFO:tensorflow:global_step/sec: 1919.23
INFO:tensorflow:loss = 0.37460858, step = 4300 (0.052 sec)
INFO:tensorflow:loss = 0.37460858, step = 4300 (0.052 sec)
INFO:tensorflow:global_step/sec: 1925.94
INFO:tensorflow:global_step/sec: 1925.94
INFO:tensorflow:loss = 0.511979, step = 4400 (0.052 sec)
INFO:tensorflow:loss = 0.511979, step = 4400 (0.052 sec)
INFO:tensorflow:global_step/sec: 1933.33
INFO:tensorflow:global_step/sec: 1933.33
INFO:tensorflow:loss = 0.01807601, step = 4500 (0.052 sec)
INFO:tensorflow:loss = 0.01807601, step = 4500 (0.052 sec)
INFO:tensorflow:global_step/sec: 2023.4
INFO:tensorflow:global_step/sec: 2023.4
INFO:tensorflow:loss = 0.30498773, step = 4600 (0.049 sec)
INFO:tensorflow:loss = 0.30498773, step = 4600 (0.049 sec)
INFO:tensorflow:global_step/sec: 1913.8
INFO:tensorflow:global_step/sec: 1913.8
INFO:tensorflow:loss = 0.30013862, step = 4700 (0.052 sec)
INFO:tensorflow:loss = 0.30013862, step = 4700 (0.052 sec)
INFO:tensorflow:global_step/sec: 2037.61
INFO:tensorflow:global_step/sec: 2037.61
INFO:tensorflow:loss = 0.2524454, step = 4800 (0.049 sec)
INFO:tensorflow:loss = 0.2524454, step = 4800 (0.049 sec)
INFO:tensorflow:global_step/sec: 2060.83
INFO:tensorflow:global_step/sec: 2060.83
INFO:tensorflow:loss = 0.2248443, step = 4900 (0.048 sec)
INFO:tensorflow:loss = 0.2248443, step = 4900 (0.048 sec)
INFO:tensorflow:global_step/sec: 1998.21
INFO:tensorflow:global_step/sec: 1998.21
INFO:tensorflow:loss = 0.13730438, step = 5000 (0.050 sec)
INFO:tensorflow:loss = 0.13730438, step = 5000 (0.050 sec)
INFO:tensorflow:global_step/sec: 2068.03
INFO:tensorflow:global_step/sec: 2068.03
INFO:tensorflow:loss = 0.0097431615, step = 5100 (0.048 sec)
INFO:tensorflow:loss = 0.0097431615, step = 5100 (0.048 sec)
INFO:tensorflow:global_step/sec: 2121.97
INFO:tensorflow:global_step/sec: 2121.97
INFO:tensorflow:loss = 0.04345341, step = 5200 (0.047 sec)
INFO:tensorflow:loss = 0.04345341, step = 5200 (0.047 sec)
INFO:tensorflow:global_step/sec: 2051.71
INFO:tensorflow:global_step/sec: 2051.71
INFO:tensorflow:loss = 0.27216375, step = 5300 (0.049 sec)
INFO:tensorflow:loss = 0.27216375, step = 5300 (0.049 sec)
INFO:tensorflow:global_step/sec: 2070.39
INFO:tensorflow:global_step/sec: 2070.39
INFO:tensorflow:loss = 0.042003, step = 5400 (0.048 sec)
INFO:tensorflow:loss = 0.042003, step = 5400 (0.048 sec)
INFO:tensorflow:global_step/sec: 1986.3
INFO:tensorflow:global_step/sec: 1986.3
INFO:tensorflow:loss = 0.056521147, step = 5500 (0.050 sec)
INFO:tensorflow:loss = 0.056521147, step = 5500 (0.050 sec)
INFO:tensorflow:global_step/sec: 2026.74
INFO:tensorflow:global_step/sec: 2026.74
INFO:tensorflow:loss = 0.107332416, step = 5600 (0.049 sec)
INFO:tensorflow:loss = 0.107332416, step = 5600 (0.049 sec)
INFO:tensorflow:global_step/sec: 2112.21
INFO:tensorflow:global_step/sec: 2112.21
INFO:tensorflow:loss = 0.04123502, step = 5700 (0.047 sec)
INFO:tensorflow:loss = 0.04123502, step = 5700 (0.047 sec)
INFO:tensorflow:global_step/sec: 1960.51
INFO:tensorflow:global_step/sec: 1960.51
INFO:tensorflow:loss = 0.07338292, step = 5800 (0.051 sec)
INFO:tensorflow:loss = 0.07338292, step = 5800 (0.051 sec)
INFO:tensorflow:global_step/sec: 2059.9
INFO:tensorflow:global_step/sec: 2059.9
INFO:tensorflow:loss = 0.24629912, step = 5900 (0.048 sec)
INFO:tensorflow:loss = 0.24629912, step = 5900 (0.048 sec)
INFO:tensorflow:global_step/sec: 1967.66
INFO:tensorflow:global_step/sec: 1967.66
INFO:tensorflow:loss = 0.015422126, step = 6000 (0.051 sec)
INFO:tensorflow:loss = 0.015422126, step = 6000 (0.051 sec)
INFO:tensorflow:global_step/sec: 2105.21
INFO:tensorflow:global_step/sec: 2105.21
INFO:tensorflow:loss = 0.02205166, step = 6100 (0.048 sec)
INFO:tensorflow:loss = 0.02205166, step = 6100 (0.048 sec)
INFO:tensorflow:global_step/sec: 2057.95
INFO:tensorflow:global_step/sec: 2057.95
INFO:tensorflow:loss = 0.0031883994, step = 6200 (0.049 sec)
INFO:tensorflow:loss = 0.0031883994, step = 6200 (0.049 sec)
INFO:tensorflow:global_step/sec: 2039.86
INFO:tensorflow:global_step/sec: 2039.86
INFO:tensorflow:loss = 0.34405905, step = 6300 (0.049 sec)
INFO:tensorflow:loss = 0.34405905, step = 6300 (0.049 sec)
INFO:tensorflow:global_step/sec: 2056.38
INFO:tensorflow:global_step/sec: 2056.38
INFO:tensorflow:loss = 0.04320626, step = 6400 (0.049 sec)
INFO:tensorflow:loss = 0.04320626, step = 6400 (0.049 sec)
INFO:tensorflow:global_step/sec: 1959.83
INFO:tensorflow:global_step/sec: 1959.83
INFO:tensorflow:loss = 0.0003893703, step = 6500 (0.051 sec)
INFO:tensorflow:loss = 0.0003893703, step = 6500 (0.051 sec)
INFO:tensorflow:global_step/sec: 2005.69
INFO:tensorflow:global_step/sec: 2005.69
INFO:tensorflow:loss = 0.005372245, step = 6600 (0.050 sec)
INFO:tensorflow:loss = 0.005372245, step = 6600 (0.050 sec)
INFO:tensorflow:global_step/sec: 1958.52
INFO:tensorflow:global_step/sec: 1958.52
INFO:tensorflow:loss = 0.017107084, step = 6700 (0.051 sec)
INFO:tensorflow:loss = 0.017107084, step = 6700 (0.051 sec)
INFO:tensorflow:global_step/sec: 1890.68
INFO:tensorflow:global_step/sec: 1890.68
INFO:tensorflow:loss = 0.00046630704, step = 6800 (0.053 sec)
INFO:tensorflow:loss = 0.00046630704, step = 6800 (0.053 sec)
INFO:tensorflow:global_step/sec: 2033.63
INFO:tensorflow:global_step/sec: 2033.63
INFO:tensorflow:loss = 0.009309545, step = 6900 (0.049 sec)
INFO:tensorflow:loss = 0.009309545, step = 6900 (0.049 sec)
INFO:tensorflow:global_step/sec: 2021.11
INFO:tensorflow:global_step/sec: 2021.11
INFO:tensorflow:loss = 0.032709546, step = 7000 (0.049 sec)
INFO:tensorflow:loss = 0.032709546, step = 7000 (0.049 sec)
INFO:tensorflow:global_step/sec: 1988.39
INFO:tensorflow:global_step/sec: 1988.39
INFO:tensorflow:loss = 0.001911474, step = 7100 (0.051 sec)
INFO:tensorflow:loss = 0.001911474, step = 7100 (0.051 sec)
INFO:tensorflow:global_step/sec: 1954.84
INFO:tensorflow:global_step/sec: 1954.84
INFO:tensorflow:loss = 0.012432969, step = 7200 (0.051 sec)
INFO:tensorflow:loss = 0.012432969, step = 7200 (0.051 sec)
INFO:tensorflow:global_step/sec: 1982.94
INFO:tensorflow:global_step/sec: 1982.94
INFO:tensorflow:loss = 0.07940982, step = 7300 (0.050 sec)
INFO:tensorflow:loss = 0.07940982, step = 7300 (0.050 sec)
INFO:tensorflow:global_step/sec: 2019.72
INFO:tensorflow:global_step/sec: 2019.72
INFO:tensorflow:loss = 0.0011171552, step = 7400 (0.049 sec)
INFO:tensorflow:loss = 0.0011171552, step = 7400 (0.049 sec)
INFO:tensorflow:global_step/sec: 1967.88
INFO:tensorflow:global_step/sec: 1967.88
INFO:tensorflow:loss = 0.010894721, step = 7500 (0.051 sec)
INFO:tensorflow:loss = 0.010894721, step = 7500 (0.051 sec)
INFO:tensorflow:global_step/sec: 1954.85
INFO:tensorflow:global_step/sec: 1954.85
INFO:tensorflow:loss = 0.0873113, step = 7600 (0.051 sec)
INFO:tensorflow:loss = 0.0873113, step = 7600 (0.051 sec)
INFO:tensorflow:global_step/sec: 1990.32
INFO:tensorflow:global_step/sec: 1990.32
INFO:tensorflow:loss = 0.009088446, step = 7700 (0.050 sec)
INFO:tensorflow:loss = 0.009088446, step = 7700 (0.050 sec)
INFO:tensorflow:global_step/sec: 1945.23
INFO:tensorflow:global_step/sec: 1945.23
INFO:tensorflow:loss = 0.009189545, step = 7800 (0.052 sec)
INFO:tensorflow:loss = 0.009189545, step = 7800 (0.052 sec)
INFO:tensorflow:global_step/sec: 2099.03
INFO:tensorflow:global_step/sec: 2099.03
INFO:tensorflow:loss = 8.943558e-05, step = 7900 (0.047 sec)
INFO:tensorflow:loss = 8.943558e-05, step = 7900 (0.047 sec)
INFO:tensorflow:global_step/sec: 2060.79
INFO:tensorflow:global_step/sec: 2060.79
INFO:tensorflow:loss = 0.012509959, step = 8000 (0.048 sec)
INFO:tensorflow:loss = 0.012509959, step = 8000 (0.048 sec)
INFO:tensorflow:global_step/sec: 2041.15
INFO:tensorflow:global_step/sec: 2041.15
INFO:tensorflow:loss = 0.014975408, step = 8100 (0.049 sec)
INFO:tensorflow:loss = 0.014975408, step = 8100 (0.049 sec)
INFO:tensorflow:global_step/sec: 2031.03
INFO:tensorflow:global_step/sec: 2031.03
INFO:tensorflow:loss = 1.5024917e-05, step = 8200 (0.049 sec)
INFO:tensorflow:loss = 1.5024917e-05, step = 8200 (0.049 sec)
INFO:tensorflow:global_step/sec: 2023.02
INFO:tensorflow:global_step/sec: 2023.02
INFO:tensorflow:loss = 0.010492004, step = 8300 (0.050 sec)
INFO:tensorflow:loss = 0.010492004, step = 8300 (0.050 sec)
INFO:tensorflow:global_step/sec: 1968.74
INFO:tensorflow:global_step/sec: 1968.74
INFO:tensorflow:loss = 0.00027322883, step = 8400 (0.051 sec)
INFO:tensorflow:loss = 0.00027322883, step = 8400 (0.051 sec)
INFO:tensorflow:global_step/sec: 2007.71
INFO:tensorflow:global_step/sec: 2007.71
INFO:tensorflow:loss = 1.0195827e-06, step = 8500 (0.050 sec)
INFO:tensorflow:loss = 1.0195827e-06, step = 8500 (0.050 sec)
INFO:tensorflow:global_step/sec: 2054.11
INFO:tensorflow:global_step/sec: 2054.11
INFO:tensorflow:loss = 0.020234762, step = 8600 (0.048 sec)
INFO:tensorflow:loss = 0.020234762, step = 8600 (0.048 sec)
INFO:tensorflow:global_step/sec: 2057.91
INFO:tensorflow:global_step/sec: 2057.91
INFO:tensorflow:loss = 0.005651293, step = 8700 (0.049 sec)
INFO:tensorflow:loss = 0.005651293, step = 8700 (0.049 sec)
INFO:tensorflow:global_step/sec: 2008.43
INFO:tensorflow:global_step/sec: 2008.43
INFO:tensorflow:loss = 0.005406373, step = 8800 (0.050 sec)
INFO:tensorflow:loss = 0.005406373, step = 8800 (0.050 sec)
INFO:tensorflow:global_step/sec: 2057.19
INFO:tensorflow:global_step/sec: 2057.19
INFO:tensorflow:loss = 0.0052554747, step = 8900 (0.049 sec)
INFO:tensorflow:loss = 0.0052554747, step = 8900 (0.049 sec)
INFO:tensorflow:global_step/sec: 2163.99
INFO:tensorflow:global_step/sec: 2163.99
INFO:tensorflow:loss = 0.005111583, step = 9000 (0.046 sec)
INFO:tensorflow:loss = 0.005111583, step = 9000 (0.046 sec)
INFO:tensorflow:global_step/sec: 2063.64
INFO:tensorflow:global_step/sec: 2063.64
INFO:tensorflow:loss = 3.581464e-05, step = 9100 (0.048 sec)
INFO:tensorflow:loss = 3.581464e-05, step = 9100 (0.048 sec)
INFO:tensorflow:global_step/sec: 2132.11
INFO:tensorflow:global_step/sec: 2132.11
INFO:tensorflow:loss = 0.07763314, step = 9200 (0.047 sec)
INFO:tensorflow:loss = 0.07763314, step = 9200 (0.047 sec)
INFO:tensorflow:global_step/sec: 2076.11
INFO:tensorflow:global_step/sec: 2076.11
INFO:tensorflow:loss = 0.07831412, step = 9300 (0.048 sec)
INFO:tensorflow:loss = 0.07831412, step = 9300 (0.048 sec)
INFO:tensorflow:global_step/sec: 2046.96
INFO:tensorflow:global_step/sec: 2046.96
INFO:tensorflow:loss = 0.0043091094, step = 9400 (0.049 sec)
INFO:tensorflow:loss = 0.0043091094, step = 9400 (0.049 sec)
INFO:tensorflow:global_step/sec: 2079.31
INFO:tensorflow:global_step/sec: 2079.31
INFO:tensorflow:loss = 0.00083986373, step = 9500 (0.048 sec)
INFO:tensorflow:loss = 0.00083986373, step = 9500 (0.048 sec)
INFO:tensorflow:global_step/sec: 2015.72
INFO:tensorflow:global_step/sec: 2015.72
INFO:tensorflow:loss = 0.0040194714, step = 9600 (0.050 sec)
INFO:tensorflow:loss = 0.0040194714, step = 9600 (0.050 sec)
INFO:tensorflow:global_step/sec: 2127.03
INFO:tensorflow:global_step/sec: 2127.03
INFO:tensorflow:loss = 0.003948324, step = 9700 (0.047 sec)
INFO:tensorflow:loss = 0.003948324, step = 9700 (0.047 sec)
INFO:tensorflow:global_step/sec: 2034.76
INFO:tensorflow:global_step/sec: 2034.76
INFO:tensorflow:loss = 0.0039140303, step = 9800 (0.049 sec)
INFO:tensorflow:loss = 0.0039140303, step = 9800 (0.049 sec)
INFO:tensorflow:global_step/sec: 2023.84
INFO:tensorflow:global_step/sec: 2023.84
INFO:tensorflow:loss = 0.0021532348, step = 9900 (0.049 sec)
INFO:tensorflow:loss = 0.0021532348, step = 9900 (0.049 sec)
INFO:tensorflow:Saving checkpoints for 10000 into models/estimator-for-XOR/model.ckpt.
INFO:tensorflow:Saving checkpoints for 10000 into models/estimator-for-XOR/model.ckpt.
INFO:tensorflow:Loss for final step: 0.00016663199.
INFO:tensorflow:Loss for final step: 0.00016663199.
<tensorflow_estimator.python.estimator.estimator.EstimatorV2 at 0x7fe829ea0f28>
my_estimator.evaluate(
input_fn=lambda: eval_input_fn(x_valid, y_valid, batch_size))
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2019-10-29T21:23:01Z
INFO:tensorflow:Starting evaluation at 2019-10-29T21:23:01Z
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from models/estimator-for-XOR/model.ckpt-10000
INFO:tensorflow:Restoring parameters from models/estimator-for-XOR/model.ckpt-10000
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Finished evaluation at 2019-10-29-21:23:01
INFO:tensorflow:Finished evaluation at 2019-10-29-21:23:01
INFO:tensorflow:Saving dict for global step 10000: binary_accuracy = 0.96, global_step = 10000, loss = 0.081909806
INFO:tensorflow:Saving dict for global step 10000: binary_accuracy = 0.96, global_step = 10000, loss = 0.081909806
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 10000: models/estimator-for-XOR/model.ckpt-10000
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 10000: models/estimator-for-XOR/model.ckpt-10000
{'binary_accuracy': 0.96, 'loss': 0.081909806, 'global_step': 10000}
...
...
Readers may ignore the next cell.
! python ../.convert_notebook_to_script.py --input ch14_part3.ipynb --output ch14_part3.py
[NbConvertApp] Converting notebook ch14_part3.ipynb to script [NbConvertApp] Writing 5044 bytes to ch14_part3.py