teratail
回答率
85.48%
質問するログイン新規登録
トップに関する質問CNN-Autoencoderでlossがnanばかり出て、Accuracyも出ずに困っています。

編集履歴

質問編集履歴

2

タグの追加

2019/03/15 10:54

投稿

yusukee345
yusukee345

スコア31

test CHANGED
File without changes
test CHANGED
File without changes

1

ソースコードの簡略化、実行環境追記

2019/03/15 10:54

投稿

yusukee345
yusukee345

スコア31

test CHANGED
@@ -1 +1 @@
1
- CNN-Autoencoderでlossがnanばかり出て、Accuracyも出ない
1
+ CNN-Autoencoderでlossがnanばかり出て、Accuracyも出ずに困っています。
test CHANGED
@@ -1,36 +1,60 @@
1
1
  失礼します。
2
2
 
3
- CNN-Autoencoderを用いた画像分類器を作ろうとしていて、folder1内の28*28画像(460枚)を読み込んでCNN-Autoencoderで学習させて、更にfolder1内の画像を用いてテストした後にテスト画像の復元(decoded)を出力しようとしています。以下の様にlossやAccuracyの値が出ません。更には復元画像もほとんど真っ黒か真っ白な画像しか出ず、明らかに学習できていません。
3
+ CNN-Autoencoderを用いた画像分類器を作ろうとしていて、folder1内の28*28画像(460枚)を読み込んでCNN-Autoencoderで学習させて、更にfolder1内の画像を用いてテストした後にテスト画像の復元(decoded)を出力しようとしています。学習を進めていくと以下の様にlossやAccuracyの値が出ません。更には復元画像もほとんど真っ白な画像しか出ず、明らかに学習できていません。
4
4
 
5
5
  ```
6
6
 
7
- step, loss, accuracy = 1: -204.758 0.000
8
-
9
- step, loss, accuracy = 2: -734.519 0.000
10
-
11
- step, loss, accuracy = 3: nan 0.957
7
+ step, loss, accuracy = 1: nan 0.935
8
+
12
-
9
+ step, loss, accuracy = 2: nan 0.935
10
+
11
+ step, loss, accuracy = 3: nan 0.891
12
+
13
- step, loss, accuracy = 4: nan 0.848
13
+ step, loss, accuracy = 4: nan 0.891
14
14
 
15
15
  step, loss, accuracy = 5: nan 0.913
16
16
 
17
+ step, loss, accuracy = 6: nan 0.978
18
+
17
- step, loss, accuracy = 6: nan 0.913
19
+ step, loss, accuracy = 7: nan 0.913
20
+
18
-
21
+ step, loss, accuracy = 8: nan 0.935
22
+
19
- step, loss, accuracy = 7: nan 0.848
23
+ step, loss, accuracy = 9: nan 0.848
20
-
21
- step, loss, accuracy = 8: nan 0.848
24
+
22
-
23
- step, loss, accuracy = 9: nan 0.891
25
+ step, loss, accuracy = 10: nan 0.913
26
+
24
-
27
+ step, loss, accuracy = 11: nan 0.935
28
+
25
- step, loss, accuracy = 10: nan 0.870
29
+ step, loss, accuracy = 12: nan 0.870
30
+
31
+ step, loss, accuracy = 13: nan 0.848
32
+
33
+ step, loss, accuracy = 14: nan 0.913
34
+
35
+ step, loss, accuracy = 15: nan 0.935
36
+
37
+ step, loss, accuracy = 16: nan 0.935
38
+
39
+ step, loss, accuracy = 17: nan 0.913
40
+
41
+ step, loss, accuracy = 18: nan 0.913
42
+
43
+ step, loss, accuracy = 19: nan 0.913
44
+
45
+ step, loss, accuracy = 20: nan 0.826
46
+
47
+ loss (test) = nan
48
+
49
+ accuracy(test) = 0.9347826
26
50
 
27
51
  ```
28
52
 
29
53
 
30
54
 
31
- 以下のコードを使用しています。my_nn_lib.pyでconvolution層等を定義したうえで、cnn_ae.pyで実行するという構造になっています。
55
+ 以下のコードを使用しています。
32
-
56
+
33
- なお、コードはこの方 https://qiita.com/TomokIshii/items/26b7414bdb3cd3052786 のコードを使わせて頂きました。my_nn_lib.pyは同一のコードを拝借しております。
57
+ コードはこの方 https://qiita.com/TomokIshii/items/26b7414bdb3cd3052786 のコードを使わせて頂きました。ネットワークの定義はより高レベルに書き換えました(tf.nn.conv2dでなくtf.layers.conv2dを使用)。
34
58
 
35
59
 
36
60
 
@@ -38,14 +62,6 @@
38
62
 
39
63
  ```
40
64
 
41
- from __future__ import absolute_import
42
-
43
- from __future__ import division
44
-
45
- from __future__ import print_function
46
-
47
-
48
-
49
65
  import numpy as np
50
66
 
51
67
  import matplotlib as mpl
@@ -82,94 +98,6 @@
82
98
 
83
99
 
84
100
 
85
-
86
-
87
- # Up-sampling 2-D Layer (deconvolutoinal Layer)
88
-
89
- class Conv2Dtranspose(object):
90
-
91
- '''
92
-
93
- constructor's args:
94
-
95
- input : input image (2D matrix)
96
-
97
- output_siz : output image size
98
-
99
- in_ch : number of incoming image channel
100
-
101
- out_ch : number of outgoing image channel
102
-
103
- patch_siz : filter(patch) size
104
-
105
- '''
106
-
107
-
108
-
109
- def __init__(self, input, output_siz, in_ch, out_ch, patch_siz, activation='relu'):
110
-
111
- self.input = input
112
-
113
- self.rows = output_siz[0]
114
-
115
- self.cols = output_siz[1]
116
-
117
- self.out_ch = out_ch
118
-
119
- self.activation = activation
120
-
121
-
122
-
123
- wshape = [patch_siz[0], patch_siz[1], out_ch, in_ch] # note the arguments order
124
-
125
-
126
-
127
- w_cvt = tf.Variable(tf.truncated_normal(wshape, stddev=0.1),
128
-
129
- trainable=True)
130
-
131
- b_cvt = tf.Variable(tf.constant(0.1, shape=[out_ch]),
132
-
133
- trainable=True)
134
-
135
- self.batsiz = tf.shape(input)[0]
136
-
137
- self.w = w_cvt
138
-
139
- self.b = b_cvt
140
-
141
- self.params = [self.w, self.b]
142
-
143
-
144
-
145
- def output(self):
146
-
147
- shape4D = [self.batsiz, self.rows, self.cols, self.out_ch]
148
-
149
- linout = tf.nn.conv2d_transpose(self.input, self.w, output_shape=shape4D,
150
-
151
- strides=[1, 2, 2, 1], padding='SAME') + self.b
152
-
153
- if self.activation == 'relu':
154
-
155
- self.output = tf.nn.relu(linout)
156
-
157
- elif self.activation == 'sigmoid':
158
-
159
- self.output = tf.sigmoid(linout)
160
-
161
- else:
162
-
163
- self.output = linout
164
-
165
-
166
-
167
- return self.output
168
-
169
-
170
-
171
-
172
-
173
101
  # Create the model
174
102
 
175
103
  def mk_nn_model(x):
@@ -178,79 +106,49 @@
178
106
 
179
107
  x_image = tf.reshape(x, [-1, image_size, image_size, 1])
180
108
 
109
+
110
+
181
- conv1 = Convolution2D(x_image, (image_size, image_size), 1, 16,
111
+ conv1 = tf.layers.conv2d(inputs=x_image, filters=16, kernel_size=(3, 3), padding='same', activation=tf.nn.relu)
182
-
183
- (3, 3), activation='relu')
112
+
184
-
185
- conv1_out = conv1.output()
186
-
187
-
188
-
189
- pool1 = MaxPooling2D(conv1_out)
190
-
191
- pool1_out = pool1.output()
192
-
193
-
194
-
195
- conv2 = Convolution2D(pool1_out, (one_layer, one_layer), 16, 8,
196
-
197
- (3, 3), activation='relu')
198
-
199
- conv2_out = conv2.output()
200
-
201
-
202
-
203
- pool2 = MaxPooling2D(conv2_out)
204
-
205
- pool2_out = pool2.output()
206
-
207
-
208
-
209
- conv3 = Convolution2D(pool2_out, (two_layer, two_layer), 8, 8, (3, 3), activation='relu')
113
+ pool1 = tf.layers.max_pooling2d(conv1, pool_size=(2, 2), strides=(2, 2), padding='same')
210
-
114
+
115
+
116
+
211
- conv3_out = conv3.output()
117
+ conv2 = tf.layers.conv2d(inputs=pool1, filters=8, kernel_size=(3, 3), padding='same', activation=tf.nn.relu)
212
-
213
-
214
-
118
+
215
- pool3 = MaxPooling2D(conv3_out)
119
+ pool2 = tf.layers.max_pooling2d(conv2, pool_size=(2, 2), strides=(2, 2), padding='same')
120
+
121
+
122
+
216
-
123
+ conv3 = tf.layers.conv2d(pool2, filters=8, kernel_size=(3, 3), padding='same', activation=tf.nn.relu)
124
+
217
- pool3_out = pool3.output()
125
+ encoded = tf.layers.max_pooling2d(conv3, pool_size=(2, 2), strides=(2, 2), padding='same')
218
126
 
219
127
  # at this point the representation is (8, 4, 4) i.e. 128-dimensional
220
128
 
221
129
  # Decoding phase
222
130
 
223
- conv_t1 = Conv2Dtranspose(pool3_out, (two_layer, two_layer), 8, 8,
131
+ upsample1 = tf.image.resize_images(encoded, size=(two_layer, two_layer), method=tf.image.ResizeMethod.NEAREST_NEIGHBOR)
224
-
132
+
225
- (3, 3), activation='relu')
133
+ conv4 = tf.layers.conv2d(inputs=upsample1, filters=8, kernel_size=(3, 3), padding='same', activation=tf.nn.relu)
226
-
227
- conv_t1_out = conv_t1.output()
134
+
228
-
229
-
230
-
135
+
136
+
231
- conv_t2 = Conv2Dtranspose(conv_t1_out, (one_layer, one_layer), 8, 8,
137
+ upsample2 = tf.image.resize_images(conv4, size=(one_layer, one_layer), method=tf.image.ResizeMethod.NEAREST_NEIGHBOR)
232
-
233
- (3, 3), activation='relu')
138
+
234
-
235
- conv_t2_out = conv_t2.output()
236
-
237
-
238
-
239
- conv_t3 = Conv2Dtranspose(conv_t2_out, (image_size, image_size), 8, 16,
139
+ conv5 = tf.layers.conv2d(inputs=upsample2, filters=8, kernel_size=(3, 3), padding='same', activation=tf.nn.relu)
240
-
241
- (3, 3), activation='relu')
140
+
242
-
243
- conv_t3_out = conv_t3.output()
141
+
244
-
245
-
246
-
142
+
247
- conv_last = Convolution2D(conv_t3_out, (image_size, image_size), 16, 1, (3, 3),
143
+ upsample3 = tf.image.resize_images(conv5, size=(image_size, image_size), method=tf.image.ResizeMethod.NEAREST_NEIGHBOR)
144
+
248
-
145
+ conv6 = tf.layers.conv2d(inputs=upsample3, filters=16, kernel_size=(3, 3), padding='same', activation=tf.nn.relu)
146
+
147
+
148
+
249
- activation='sigmoid')
149
+ logits = tf.layers.conv2d(inputs=conv6, filters=1, kernel_size=(3, 3), padding='same', activation=None)
250
-
150
+
251
- decoded = conv_last.output()
151
+ decoded = tf.nn.sigmoid(logits)
252
-
253
-
254
152
 
255
153
  decoded = tf.reshape(decoded, [-1, image_size*image_size])
256
154
 
@@ -310,7 +208,7 @@
310
208
 
311
209
 
312
210
 
313
- epochs = 10
211
+ epochs = 20
314
212
 
315
213
  batch_size = data_number // 10
316
214
 
@@ -346,11 +244,11 @@
346
244
 
347
245
  test_image = X[0:batch_size]
348
246
 
349
- decoded_imgs = decoded.eval(test_image)
247
+ decoded_imgs, test_loss, test_accuracy = sess.run([decoded, loss, accuracy], feed_dict={x: test_image})
350
-
248
+
351
- print('loss (test) = ', loss.eval(test_image))
249
+ print('loss (test) = ', test_loss)
352
-
250
+
353
- print('accuracy(test) = ', accuracy.eval(test_image))
251
+ print('accuracy(test) = ', test_accuracy)
354
252
 
355
253
 
356
254
 
@@ -398,6 +296,16 @@
398
296
 
399
297
  ```
400
298
 
299
+ ####実行環境
300
+
301
+ OS Windows10 64bit
302
+
303
+ プロセッサ　Intel(R) Core(TM)i7-8700k CPU @ 3.70GHz
304
+
305
+ RAM 32.0 GB
306
+
307
+ Anaconda Prompt
308
+
401
309
 
402
310
 
403
311
  ####試したこと