Tensorflowで異なるセッションでのrestoreして値の復元する際に、uninitializedと初期化しているのに表示される

#savar.save()で保存した値を異なるセッションでrestore()する際に初期化しているにも関わらず初期化されていない(uninitialize)とエラーが表示される。
45個のデータを学習して、restoreを行って新しいsessionの中で１個分の実践データを吐き出すプログラムを作っています。

エラーコード

FailedPreconditionError (see above for traceback): Attempting to use uninitialized value hidden1_1/hidden1_bias
         [[Node: hidden1_1/hidden1_bias/read = Identity[T=DT_FLOAT, _class=["loc:@hidden1_1/hidden1_bias"], _device="/job:localhost/replica:0/task:0/cpu:0"](hidden1_1/hidden1_bias)]]

#プログラムのフロー
1,session1の中で学習時の重み・バイアスをsavar.save()を使ってckptファイルを作りdiskに保存する。
※model.ckpt.metaファイルを見ても変数名は書かれているものの、重み・バイアスの値は書かれておらず、diskに保存されている→ckpt.metaファイルをもとに後ろで動いているC++で復元しているのか（脱線）
→それで納得する。
tf.train.Saver
2,session2の中で新しく、output2 = inference(score_placeholder)を定義する。
※１新たに定義するのはsession2の入力データが1セットのみのデータなので、正規化する式をif~elseで避けているためです。
※２inference()はデータを入力層―隠れ層―出力層を通して予想値を計算する関数
※３print()関数はpythonで実行されるも、th.l2_nomalize,malmulなどはsession(run)でC++に渡されて実行される。
3,output2に含まれる（output_bias,output_weight,hidden_bias,hidden_weight）の値がuninitializeの状態？
※ここの値はまだrestoreしていないので、形式？だけは入っている状態（これがいけないってこと？）
→ここの理解が不足している、run and designの形式とう学ぶ必要あり。

def inference(score_placeholder):
  with tf.name_scope('hidden1') as scope:
    hidden1_weight = tf.Variable(tf.truncated_normal([SCORE_SIZE, HIDDEN_UNIT_SIZE], stddev=0.01), name="hidden1_weight")
    hidden1_bias = tf.Variable(tf.constant(0.1, shape=[HIDDEN_UNIT_SIZE]), name="hidden1_bias")
    hidden1_output = tf.nn.relu(tf.matmul(score_placeholder, hidden1_weight) + hidden1_bias)
  with tf.name_scope('output') as scope:
    output_weight = tf.Variable(tf.truncated_normal([HIDDEN_UNIT_SIZE, 1], stddev=0.01), name="output_weight")
    output_bias = tf.Variable(tf.constant(0.1, shape=[1]), name="output_bias")
    output = tf.matmul(hidden1_output, output_weight) + output_bias
    if TACK == 1:
        print("saku1")
        return output
    else :
        print("saku2")
        return tf.nn.l2_normalize(output, 0)

4,resotreを用いて重み・バイアスがsession1の状態が復元→variableの値が10000回学習した後の値になる）
5,best_match2 = sess2.run(output3, feed_dict={score_placeholder:[test2]})を実行すれば、復元した重み・バイアスで出力されるはずだとおもうのですが・・・そうならない。

という状態です。

思い当たることは

また別の問題として、
・そもそもinitializeをsessionの最初に書いても、初期化するvariableがないのに何でするのかという疑問があります。
・sessionで関数は実行されているのをみるとsessionの役割はtf.~がつくものの計算が思いから後ろで動いているC++に渡している認識で間違っていないか知りたいです。

何度もお手間を取らせてしまっていますがよろしくお願い致します。


def inference(score_placeholder):
  with tf.name_scope('hidden1') as scope:
    hidden1_weight = tf.Variable(tf.truncated_normal([SCORE_SIZE, HIDDEN_UNIT_SIZE], stddev=0.01), name="hidden1_weight")
    hidden1_bias = tf.Variable(tf.constant(0.1, shape=[HIDDEN_UNIT_SIZE]), name="hidden1_bias")
    hidden1_output = tf.nn.relu(tf.matmul(score_placeholder, hidden1_weight) + hidden1_bias)
  with tf.name_scope('output') as scope:
    output_weight = tf.Variable(tf.truncated_normal([HIDDEN_UNIT_SIZE, 1], stddev=0.01), name="output_weight")
    output_bias = tf.Variable(tf.constant(0.1, shape=[1]), name="output_bias")
    output = tf.matmul(hidden1_output, output_weight) + output_bias
    if TACK == 1:
        print("saku1")
        return output
    else :
        print("saku2")
        return tf.nn.l2_normalize(output, 0)

variable()に保存されている値を異なるセッションで、restoreする際に
output_bias,output_weight,hidden_bias,hidden_weightの値が初期化されていない旨のエラーが発生してしましました。

１、session

入力層ー隠れ層ー出力層で構成されるNN
inference関数の引数になる実践データが予測値を吐き出す認識

def inference(score_placeholder):
  with tf.name_scope('hidden1') as scope:
    hidden1_weight = tf.Variable(tf.truncated_normal([SCORE_SIZE, HIDDEN_UNIT_SIZE], stddev=0.01), name="hidden1_weight")
    hidden1_bias = tf.Variable(tf.constant(0.1, shape=[HIDDEN_UNIT_SIZE]), name="hidden1_bias")
    hidden1_output = tf.nn.relu(tf.matmul(score_placeholder, hidden1_weight) + hidden1_bias)
  with tf.name_scope('output') as scope:
    output_weight = tf.Variable(tf.truncated_normal([HIDDEN_UNIT_SIZE, 1], stddev=0.01), name="output_weight")
    output_bias = tf.Variable(tf.constant(0.1, shape=[1]), name="output_bias")
    output = tf.matmul(hidden1_output, output_weight) + output_bias
    if TACK == 1:
        print("saku1")
        return output
    else :
        print("saku2")
        return tf.nn.l2_normalize(output, 0)

 with tf.Session() as sess2
  
  saver=tf.train.Saver()
    saver.save(sess,cwd+'/model.ckpt')
 with tf.Session() as sess2

FailedPreconditionError (see above for traceback): Attempting to use uninitialized value output_1/output_bias

one_data.csv

0.71428573,  0.85714287,  0.71428573,  0.5714286 ,  0.5714286 ,0.71428573,  0.5714286 ,  0.71428573,  0.71428573,  0.71428573,0.5714286 ,  0.71428573

ソースコード

import tensorflow as tf
import numpy
import os

cwd = os.getcwd()

SCORE_SIZE = 12
HIDDEN_UNIT_SIZE = 40
TRAIN_DATA_SIZE = 45
TACK = 0
raw_input = numpy.loadtxt(open("test.csv"), delimiter=",")
[tensor, score]  = numpy.hsplit(raw_input, [1])
[tensor_train, tensor_test] = numpy.vsplit(tensor, [TRAIN_DATA_SIZE])
[score_train, score_test] = numpy.vsplit(score, [TRAIN_DATA_SIZE])
#tensorは正解データtrainは学習モデル、scoreは学習データ、testは実データ

def inference(score_placeholder):
  with tf.name_scope('hidden1') as scope:
    hidden1_weight = tf.Variable(tf.truncated_normal([SCORE_SIZE, HIDDEN_UNIT_SIZE], stddev=0.01), name="hidden1_weight")
    hidden1_bias = tf.Variable(tf.constant(0.1, shape=[HIDDEN_UNIT_SIZE]), name="hidden1_bias")
    hidden1_output = tf.nn.relu(tf.matmul(score_placeholder, hidden1_weight) + hidden1_bias)
  with tf.name_scope('output') as scope:
    output_weight = tf.Variable(tf.truncated_normal([HIDDEN_UNIT_SIZE, 1], stddev=0.01), name="output_weight")
    output_bias = tf.Variable(tf.constant(0.1, shape=[1]), name="output_bias")
    output = tf.matmul(hidden1_output, output_weight) + output_bias
    if TACK == 1:
        print("saku1")
        return output
    else :
        print("saku2")
        return tf.nn.l2_normalize(output, 0)

def loss(output, tensor_placeholder, loss_label_placeholder):
  with tf.name_scope('loss') as scope:
    loss = tf.nn.l2_loss(output - tf.nn.l2_normalize(tensor_placeholder, 0))
    tf.summary.scalar('loss_label_placeholder', loss)
  return loss

def training(loss):
  with tf.name_scope('training') as scope:
    train_step = tf.train.GradientDescentOptimizer(0.01).minimize(loss)
  return train_step



with tf.Graph().as_default():
  tensor_placeholder = tf.placeholder(tf.float32, [None, 1], name="tensor_placeholder")
  score_placeholder = tf.placeholder(tf.float32, [None, SCORE_SIZE], name="score_placeholder")
  loss_label_placeholder = tf.placeholder("string", name="loss_label_placeholder")

  feed_dict_train={
    tensor_placeholder: tensor_train,
    score_placeholder: score_train,
    loss_label_placeholder: "loss_train"
  }

  feed_dict_test={
    tensor_placeholder: tensor_test,
    score_placeholder: score_test,
    loss_label_placeholder: "loss_test"
  }

  output = inference(score_placeholder)
  loss = loss(output, tensor_placeholder, loss_label_placeholder)
  training_op = training(loss)
  summary_op =  tf.summary.merge_all()
  init = tf.global_variables_initializer()
  best_loss = float("inf")

  with tf.Session() as sess:
    summary_writer = tf.summary.FileWriter('data', graph_def=sess.graph_def)
    sess.run(init)
    for step in range(3000):
      sess.run(training_op, feed_dict=feed_dict_train)
      loss_test = sess.run(loss, feed_dict=feed_dict_test)
      if loss_test < best_loss:
        best_loss = loss_test
        best_match = sess.run(output, feed_dict=feed_dict_test)
      if step % 100 == 0:
        summary_str = sess.run(summary_op, feed_dict=feed_dict_test)
        summary_str += sess.run(summary_op, feed_dict=feed_dict_train)
        summary_writer.add_summary(summary_str, step)
    saver=tf.train.Saver()
    saver.save(sess,cwd+'/model.ckpt')
    print(cwd)
    print(best_match)
    print('Saved a model.')
    sess.close()

  with tf.Session() as sess2:
    #summary_writer = tf.summary.FileWriter('data2', graph=sess2.graph_def)
    sess2.run(init)

    TACK = 1
    saver = tf.train.Saver()
    saver.restore(sess2,cwd + "/model.ckpt")
    output3 = inference(score_placeholder)
    test2 = numpy.loadtxt(open("one_data.csv"), delimiter=",").astype(numpy.float32)
    best_match2 = sess2.run(output3, feed_dict={score_placeholder:[test2]})
    summary_str = sess2.run(summary_op, feed_dict=feed_dict_test)
    print(best_match2)
    print("fin")
    sess2.close()

行動規範の内容に同意します

回答2件

Python
1import tensorflow as tf
2import numpy
3import os
4
5cwd = os.getcwd()
6
7SCORE_SIZE = 12
8HIDDEN_UNIT_SIZE = 40
9TRAIN_DATA_SIZE = 45
10TACK = 0
11raw_input = numpy.loadtxt(open("test.csv"), delimiter=",")
12[tensor, score]  = numpy.hsplit(raw_input, [1])
13[tensor_train, tensor_test] = numpy.vsplit(tensor, [TRAIN_DATA_SIZE])
14[score_train, score_test] = numpy.vsplit(score, [TRAIN_DATA_SIZE])
15#tensorは正解データtrainは学習モデル、scoreは学習データ、testは実データ
16
17def define_variable():
18    with tf.name_scope('hidden1') as scope:
19        hidden1_weight = tf.Variable(tf.truncated_normal([SCORE_SIZE, HIDDEN_UNIT_SIZE], stddev=0.01), name="hidden1_weight")
20        hidden1_bias = tf.Variable(tf.constant(0.1, shape=[HIDDEN_UNIT_SIZE]), name="hidden1_bias")
21    with tf.name_scope('output') as scope:
22        output_weight = tf.Variable(tf.truncated_normal([HIDDEN_UNIT_SIZE, 1], stddev=0.01), name="output_weight")
23        output_bias = tf.Variable(tf.constant(0.1, shape=[1]), name="output_bias")
24    return hidden1_weight, hidden1_bias, output_weight, output_bias
25
26def inference(score_placeholder):
27  with tf.name_scope('hidden1') as scope:
28    hidden1_output = tf.nn.relu(tf.matmul(score_placeholder, hidden1_weight) + hidden1_bias)
29  with tf.name_scope('output') as scope:
30    output = tf.matmul(hidden1_output, output_weight) + output_bias
31    if TACK == 1:
32        print("saku1")
33        return output
34    else :
35        print("saku2")
36        return tf.nn.l2_normalize(output, 0)
37
38def loss(output, tensor_placeholder, loss_label_placeholder):
39  with tf.name_scope('loss') as scope:
40    loss = tf.nn.l2_loss(output - tf.nn.l2_normalize(tensor_placeholder, 0))
41    tf.summary.scalar('loss_label_placeholder', loss)
42  return loss
43
44def training(loss):
45  with tf.name_scope('training') as scope:
46    train_step = tf.train.GradientDescentOptimizer(0.01).minimize(loss)
47  return train_step
48
49
50with tf.Graph().as_default():
51  hidden1_weight, hidden1_bias, output_weight, output_bias = define_variable()
52  tensor_placeholder = tf.placeholder(tf.float32, [None, 1], name="tensor_placeholder")
53  score_placeholder = tf.placeholder(tf.float32, [None, SCORE_SIZE], name="score_placeholder")
54  loss_label_placeholder = tf.placeholder("string", name="loss_label_placeholder")
55
56  feed_dict_train={
57    tensor_placeholder: tensor_train,
58    score_placeholder: score_train,
59    loss_label_placeholder: "loss_train"
60  }
61
62  feed_dict_test={
63    tensor_placeholder: tensor_test,
64    score_placeholder: score_test,
65    loss_label_placeholder: "loss_test"
66  }
67
68  output = inference(score_placeholder)
69  loss = loss(output, tensor_placeholder, loss_label_placeholder)
70  training_op = training(loss)
71  summary_op =  tf.summary.merge_all()
72  init = tf.global_variables_initializer()
73  best_loss = float("inf")
74
75  with tf.Session() as sess:
76    summary_writer = tf.summary.FileWriter('data', graph_def=sess.graph_def)
77    sess.run(init)
78    for step in range(3000):
79      sess.run(training_op, feed_dict=feed_dict_train)
80      loss_test = sess.run(loss, feed_dict=feed_dict_test)
81      if loss_test < best_loss:
82        best_loss = loss_test
83        best_match = sess.run(output, feed_dict=feed_dict_test)
84      if step % 100 == 0:
85        summary_str = sess.run(summary_op, feed_dict=feed_dict_test)
86        summary_str += sess.run(summary_op, feed_dict=feed_dict_train)
87        summary_writer.add_summary(summary_str, step)
88    saver=tf.train.Saver()
89    saver.save(sess,cwd+'/model.ckpt')
90    print(cwd)
91    print(best_match)
92    print('Saved a model.')
93    sess.close()
94
95  with tf.Session() as sess2:
96    #summary_writer = tf.summary.FileWriter('data2', graph=sess2.graph_def)
97    TACK = 1
98    output3 = inference(score_placeholder)
99    sess2.run(init)
100
101    saver = tf.train.Saver()
102    saver.restore(sess2,cwd + "/model.ckpt")
103    test2 = numpy.loadtxt(open("one_data.csv"), delimiter=",").astype(numpy.float32)
104    best_match2 = sess2.run(output3, feed_dict={score_placeholder:[test2]})
105    summary_str = sess2.run(summary_op, feed_dict=feed_dict_test)
106    print(best_match2)
107    print("fin")
108    sess2.close()

inference()の中で hidden_bias と output_biasをprintすると以下のようになります。

<tf.Variable 'hidden1/hidden1_bias:0' shape=(40,) dtype=float32_ref>
<tf.Variable 'output/output_bias:0' shape=(1,) dtype=float32_ref>
saku2

<tf.Variable 'hidden1_1/hidden1_bias:0' shape=(40,) dtype=float32_ref>
<tf.Variable 'output_1/output_bias:0' shape=(1,) dtype=float32_ref>

hidden1, と hidden1_1、outputと output1_1 と違うnamesopeの変数になっています。したがって、
hidden1/hidden_biasは restoreによって初期化されますが、 hidden1_1/hidden_bias は初期化されていません。

variable の宣言は一回にまとめるようにすると良いでしょう。
もともと、このプログラムが学習→保存と、読込→実行が別のスクリプトであれば起きない問題ではあります。

しかし、Tensorflow は Define and Run でテンソルの演算を行います。計算グラフの定義と実行を別々に行うモデルです。このプログラムが、計算グラフの定義と実行をごちゃごちゃにしているので、わかりにくくなっていると思います。

ディープラーニングの計算を普通のプログラムと同じように書きたいのでしたら PyTorch みたいな Define by Runのライブラリが良いかもしれません。あるいは、Kerasのようなライブラリだと Define and Run を強制させられるので、そのあたりのつまづきは少ないです。

私はディープラーニング初心者の方には TensorFlowより Kerasを勧めます。

投稿2017/07/13 21:47

編集2017/07/15 05:22

MasashiKimura

総合スコア1150

使っているtest.csv(一部)です｜スレッドを見づらくして申し訳ないです。

0.571428571,0.571428571,0.714285714,0.285714286,0.571428571,0.571428571,0.571428571,0.571428571,0.428571429,0.571428571,0.571428571,0.571428571,0.571428571
0.571428571,0.285714286,0.714285714,0.857142857,0.714285714,0.285714286,0.571428571,0.428571429,0.857142857,0.714285714,0.285714286,0.285714286,1
0.285714286,1,1,1,0.571428571,0.571428571,0.571428571,0.857142857,0.857142857,0.285714286,0.571428571,0.571428571,0.571428571
0.428571429,0.857142857,0.714285714,1,0.571428571,0.428571429,0.714285714,0.571428571,0.714285714,0.571428571,0.571428571,0.857142857,0.571428571
0.571428571,0.857142857,0.714285714,0.857142857,0.714285714,0.428571429,0.714285714,0.714285714,0.571428571,0.571428571,0.714285714,0.857142857,0.857142857
0.571428571,0.714285714,0.857142857,1,0.571428571,0.571428571,0.571428571,0.714285714,0.857142857,0.714285714,0.428571429,0.571428571,0.714285714
0.714285714,0.571428571,0.714285714,0.857142857,0.714285714,0.428571429,0.571428571,0.714285714,0.714285714,0.714285714,0.285714286,0.571428571,0.714285714
0.857142857,0.857142857,0.857142857,1,0.571428571,0.571428571,0.857142857,0.857142857,0.714285714,0.857142857,0.714285714,0.714285714,0.857142857
0.285714286,0.571428571,0.571428571,0.428571429,0.571428571,0.571428571,0.571428571,0.571428571,0.571428571,0.571428571,0.571428571,0.571428571,0.285714286
0.571428571,0.714285714,0.857142857,0.714285714,0.571428571,0.571428571,0.714285714,0.571428571,0.714285714,0.714285714,0.714285714,0.571428571,0.714285714
0.428571429,0.285714286,0.714285714,0.857142857,0.428571429,0.285714286,0.428571429,0.142857143,0.714285714,0.428571429,0.428571429,0.428571429,0.714285714