In [3]:
# %load /Users/facai/Study/book_notes/preconfig.py
%matplotlib inline

import matplotlib.pyplot as plt
import seaborn as sns
sns.set(color_codes=True)
sns.set(font='SimHei', font_scale=2.5)
plt.rcParams['axes.grid'] = False

import tensorflow as tf

def show_image(filename, figsize=None, res_dir=True):
    if figsize:
        plt.figure(figsize=figsize)

    if res_dir:
        filename = './res/{}'.format(filename)

    plt.imshow(plt.imread(filename))
/usr/local/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:34: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters

TensorFlow图相关知识简介

参考: https://www.tensorflow.org/programmers_guide/graphs

  1. tf.Graph

  2. op, tensor

  3. variable

  4. name_scope, variable_scop, collection

  5. save and restore

0. tf.Graph

tf.Graph: GraphDef => *.pb文件

  • Graph structure: Operator, Tensor-like object, 连接关系
  • Graph collections: metadata

tf.Session():

  • 本地
  • 分布式:master (worker_0)
with tf.Session("grpc://example.org:2222"):
  pass

状态(Variable) => *.ckpt文件

1. 算子与Tensor

In [6]:
a = tf.constant(1)
b = a * 2
b
Out[6]:
<tf.Tensor 'mul_1:0' shape=() dtype=int32>
In [7]:
b.op
Out[7]:
<tf.Operation 'mul_1' type=Mul>
In [11]:
b.consumers()
Out[11]:
[]
In [15]:
a.op
Out[15]:
<tf.Operation 'Const_1' type=Const>
In [19]:
a.consumers()
Out[19]:
[<tf.Operation 'mul_1' type=Mul>]

tensorflow/python/framework/ops.py

  • Tensor:
    • device
    • graph
    • op
    • consumers
    • _override_operator: 数学算子:math_op.add 重载 __add__
In [8]:
b.op.outputs
Out[8]:
[<tf.Tensor 'mul_1:0' shape=() dtype=int32>]
In [9]:
list(b.op.inputs)
Out[9]:
[<tf.Tensor 'Const_1:0' shape=() dtype=int32>,
 <tf.Tensor 'mul_1/y:0' shape=() dtype=int32>]
In [14]:
print(b.op.inputs[0])
print(a)
Tensor("Const_1:0", shape=(), dtype=int32)
Tensor("Const_1:0", shape=(), dtype=int32)
In [17]:
list(a.op.inputs)
Out[17]:
[]
  • Operator: NodeDef
    • device
    • inputs
    • outputs
    • graph
    • node_def
    • op_def
    • run
    • traceback

Operator和Tensor构成无向图

# run
sess.run([b])

参考:

2. 变量

In [20]:
v = tf.Variable([0])
c = b + v
c
Out[20]:
<tf.Tensor 'add:0' shape=(1,) dtype=int32>
In [23]:
list(c.op.inputs)
Out[23]:
[<tf.Tensor 'mul_1:0' shape=() dtype=int32>,
 <tf.Tensor 'Variable/read:0' shape=(1,) dtype=int32>]
In [25]:
c.op.inputs[1].op
Out[25]:
<tf.Operation 'Variable/read' type=Identity>
In [26]:
list(c.op.inputs[1].op.inputs)
Out[26]:
[<tf.Tensor 'Variable:0' shape=(1,) dtype=int32_ref>]
In [21]:
v
Out[21]:
<tf.Variable 'Variable:0' shape=(1,) dtype=int32_ref>

实际上,对变量的读是通过tf.identity算子得到:

c = tf.add(b, tf.identity(v))
  • Variable: act like Tensor
    • ops
      1. VariableV2
      2. ResourceVariable
    • _AsTensor -> g.as_graph_element
    • value: Identity(variable) -> Tensor
    • assign
    • init_op: Assign(self, init_value)
    • to_proto: VariableDef

参考:https://www.tensorflow.org/versions/master/api_docs/python/tf/Variable

3. collections

  • collections: 按作用分组
  • name_scope: Operator, Tensor
  • variable_scope: Variable
    • 伴生name_scope
class Layer:
  def build(self):
    pass
  def call(self, inputs):
    pass

参考:https://www.tensorflow.org/programmers_guide/summaries_and_tensorboard

4. 保存与恢复

In [58]:
graph_a = tf.Graph()
with graph_a.as_default():
    v1 = tf.get_variable("v1", shape=[3], initializer = tf.zeros_initializer)
    print(v1)
    inc_v1 = v1.assign(v1+1)
      
    init_op = tf.global_variables_initializer()
    saver = tf.train.Saver()
  
    with tf.Session() as sess:
        sess.run(init_op)
        inc_v1.op.run()
        save_path = saver.save(sess, "./tmp/model.ckpt", write_meta_graph=True)
        print("Model saved in path: %s" % save_path)
        
        pb_path = tf.train.write_graph(graph_a.as_graph_def(), "./tmp/", "graph.pbtxt", as_text=True)
        print("Graph saved in path: %s" % pb_path)
<tf.Variable 'v1:0' shape=(3,) dtype=float32_ref>
Model saved in path: ./tmp/model.ckpt
Graph saved in path: ./tmp/graph.pbtxt

graph.pbtxt部份示意:v1 + 1:

node {
  name: "add"
  op: "Add"
  input: "v1/read"
  input: "add/y"
  attr {
    key: "T"
    value {
      type: DT_FLOAT
    }
  }
}
In [62]:
graph_b = tf.Graph()
with graph_b.as_default():
    with tf.Session() as sess:
        saver = tf.train.import_meta_graph('./tmp/model.ckpt.meta')
        saver.restore(sess, "./tmp/model.ckpt")
        print(graph_b.get_operations())
        
        v1 = graph_b.get_tensor_by_name("v1:0")
        print("------------------")
        print("v1 : %s" % v1.eval(session=sess))
INFO:tensorflow:Restoring parameters from ./tmp/model.ckpt
[<tf.Operation 'v1/Initializer/zeros' type=Const>, <tf.Operation 'v1' type=VariableV2>, <tf.Operation 'v1/Assign' type=Assign>, <tf.Operation 'v1/read' type=Identity>, <tf.Operation 'add/y' type=Const>, <tf.Operation 'add' type=Add>, <tf.Operation 'Assign' type=Assign>, <tf.Operation 'init' type=NoOp>, <tf.Operation 'save/Const' type=Const>, <tf.Operation 'save/SaveV2/tensor_names' type=Const>, <tf.Operation 'save/SaveV2/shape_and_slices' type=Const>, <tf.Operation 'save/SaveV2' type=SaveV2>, <tf.Operation 'save/control_dependency' type=Identity>, <tf.Operation 'save/RestoreV2/tensor_names' type=Const>, <tf.Operation 'save/RestoreV2/shape_and_slices' type=Const>, <tf.Operation 'save/RestoreV2' type=RestoreV2>, <tf.Operation 'save/Assign' type=Assign>, <tf.Operation 'save/restore_all' type=NoOp>]
------------------
v1 : [1. 1. 1.]

总结:

参考: