ValueError: Shapes (128, 128, 1) and (128, 128, 3) are incompatible

Question

閲覧いただきありがとうございます。畳み込み層を用いたオートエンコーダでタイトルのエラーが解決できなかったため質問いたしました。

扱うデータがグレースケールであるため、x_trainとx_validをreshapeで変換しました。x_trainのshapeは(128,128,1)と変換できているのに対し、x_validが(128,128,3)でありshapeが違うのが問題であるとは思うのですが、解決できませんでした。

ちなみに入力しているデータは128*128のデータが146枚となっています。

初心者の質問で申し訳ありませんがよろしくお願い致します。

エラーコード

ValueError                                Traceback (most recent call last)
<ipython-input-6-39c46d683b62> in <module>()
     62                 batch_size=32,
     63                 shuffle=False,
---> 64                 validation_data=(x_valid, x_valid))
     65 
     66 encoded_img = encoder.predict(x_train)

9 frames
/usr/local/lib/python3.7/dist-packages/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing)
   1182                 _r=1):
   1183               callbacks.on_train_batch_begin(step)
-> 1184               tmp_logs = self.train_function(iterator)
   1185               if data_handler.should_sync:
   1186                 context.async_wait()

/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/def_function.py in __call__(self, *args, **kwds)
    883 
    884       with OptionalXlaContext(self._jit_compile):
--> 885         result = self._call(*args, **kwds)
    886 
    887       new_tracing_count = self.experimental_get_tracing_count()

/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/def_function.py in _call(self, *args, **kwds)
    931       # This is the first call of __call__, so we have to initialize.
    932       initializers = []
--> 933       self._initialize(args, kwds, add_initializers_to=initializers)
    934     finally:
    935       # At this point we know that the initialization is complete (or less

/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/def_function.py in _initialize(self, args, kwds, add_initializers_to)
    758     self._concrete_stateful_fn = (
    759         self._stateful_fn._get_concrete_function_internal_garbage_collected(  # pylint: disable=protected-access
--> 760             *args, **kwds))
    761 
    762     def invalid_creator_scope(*unused_args, **unused_kwds):

/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py in _get_concrete_function_internal_garbage_collected(self, *args, **kwargs)
   3064       args, kwargs = None, None
   3065     with self._lock:
-> 3066       graph_function, _ = self._maybe_define_function(args, kwargs)
   3067     return graph_function
   3068 

/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py in _maybe_define_function(self, args, kwargs)
   3461 
   3462           self._function_cache.missed.add(call_context_key)
-> 3463           graph_function = self._create_graph_function(args, kwargs)
   3464           self._function_cache.primary[cache_key] = graph_function
   3465 

/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py in _create_graph_function(self, args, kwargs, override_flat_arg_shapes)
   3306             arg_names=arg_names,
   3307             override_flat_arg_shapes=override_flat_arg_shapes,
-> 3308             capture_by_value=self._capture_by_value),
   3309         self._function_attributes,
   3310         function_spec=self.function_spec,

/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/func_graph.py in func_graph_from_py_func(name, python_func, args, kwargs, signature, func_graph, autograph, autograph_options, add_control_dependencies, arg_names, op_return_value, collections, capture_by_value, override_flat_arg_shapes, acd_record_initial_resource_uses)
   1005         _, original_func = tf_decorator.unwrap(python_func)
   1006 
-> 1007       func_outputs = python_func(*func_args, **func_kwargs)
   1008 
   1009       # invariant: `func_outputs` contains only Tensors, CompositeTensors,

/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/def_function.py in wrapped_fn(*args, **kwds)
    666         # the function a weak reference to itself to avoid a reference cycle.
    667         with OptionalXlaContext(compile_with_xla):
--> 668           out = weak_wrapped_fn().__wrapped__(*args, **kwds)
    669         return out
    670 

/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/func_graph.py in wrapper(*args, **kwargs)
    992           except Exception as e:  # pylint:disable=broad-except
    993             if hasattr(e, "ag_error_metadata"):
--> 994               raise e.ag_error_metadata.to_exception(e)
    995             else:
    996               raise

ValueError: in user code:

    /usr/local/lib/python3.7/dist-packages/keras/engine/training.py:853 train_function  *
        return step_function(self, iterator)
    <ipython-input-6-39c46d683b62>:38 ssim_loss  *
        return 1-tf.reduce_mean(tf.image.ssim(y_true, y_pred,
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/util/dispatch.py:206 wrapper  **
        return target(*args, **kwargs)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/image_ops_impl.py:4304 ssim
        _, _, checks = _verify_compatible_image_shapes(img1, img2)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/image_ops_impl.py:3994 _verify_compatible_image_shapes
        shape1[-3:].assert_is_compatible_with(shape2[-3:])
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/tensor_shape.py:1161 assert_is_compatible_with
        raise ValueError("Shapes %s and %s are incompatible" % (self, other))

    ValueError: Shapes (128, 128, 1) and (128, 128, 3) are incompatible

ソースコード

from keras.layers import Input, Conv2D, BatchNormalization, Activation, MaxPooling2D, UpSampling2D, Reshape
from keras.models import Model
from keras.preprocessing.image import load_img, save_img, img_to_array, array_to_img
import os
import glob
from PIL import Image
import numpy as np
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt
from google.colab import drive
import sunpy.map

drive.mount('/content/drive', force_remount=True)

input_data = np.load('drive/My Drive/study/temp_201005_AR8_resize_128128.npy')
t = np.zeros(input_data.shape[0],dtype='int32')

input_img = Input(shape=(128, 128,))
hidden = Reshape((128, 128, 1), input_shape = (128, 128))(input_img)
x = Conv2D(64, (3, 3), padding='same')(hidden)
x = BatchNormalization()(x)
x = Activation('relu')(x)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(32, (3, 3), padding='same')(x)
x = BatchNormalization()(x)
x = Activation('relu')(x)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(16, (3, 3), padding='same')(x)
x = BatchNormalization()(x)
x = Activation('relu')(x)
encoded = MaxPooling2D((2, 2), padding='same')(x)

x = Conv2D(16, (3, 3), padding='same')(encoded)
x = BatchNormalization()(x)
x = Activation('relu')(x)
x = UpSampling2D((2, 2))(x)
x = Conv2D(32, (3, 3), padding='same')(x)
x = BatchNormalization()(x)
x = Activation('relu')(x)
x = UpSampling2D((2, 2))(x)
x = Conv2D(64, (3, 3), padding='same')(x)
x = BatchNormalization()(x)
x = Activation('relu')(x)
x = UpSampling2D((2, 2))(x)
x = Conv2D(3, (3, 3), padding='same')(x)
x = BatchNormalization()(x)
decoded = Activation('sigmoid')(x)

autoencoder = Model(input_img, decoded)

from keras import optimizers
import tensorflow as tf

def ssim_loss(y_true, y_pred):
    return 1-tf.reduce_mean(tf.image.ssim(y_true, y_pred, 
                                          max_val = 1.0,filter_size=11,
                                        filter_sigma=1.5, k1=0.01, k2=0.03 ))

adam = tf.keras.optimizers.Adam(learning_rate=0.001, beta_1=0.9, beta_2=0.999, epsilon=None, decay=0.0, amsgrad=False)
autoencoder.compile(optimizer=adam, loss=ssim_loss)

x_train_valid, x_test, t_train_valid, t_test = train_test_split(input_data, t, test_size=0.1, shuffle=42)
x_train, x_valid, t_train, t_valid = train_test_split(x_train_valid, t_train_valid, test_size=0.1)
x_train = (x_train.astype('float32')+2000)/4000.
x_valid = (x_valid.astype('float32')+2000)/4000.
x_test = (x_test.astype('float32')+2000)/4000.

x_train = x_train.reshape((-1, 128, 128, 1))
x_valid = x_valid.reshape((-1, 128, 128, 1))
#x_test = x_test.reshape((-1, 128, 128, 1))

print(x_train.shape)
print(x_valid.shape)
print(x_test.shape)

# run autoencoder
hist = autoencoder.fit(x_train, x_train,
                epochs=20,
                batch_size=32,
                shuffle=False,
                validation_data=(x_valid, x_valid)) ##ここでエラーが出る

encoded_img = encoder.predict(x_train)
decoded_img = decoder.predict(encoded_img)

Comments

[jbpb0]

> 入力しているデータは128*128のデータが146枚

input_data = np.load(...
のすぐ次に
print(input_data.shape)
を追加して実行したら、その結果は
(146, 128, 128)
と表示されるのでしょうか？

[jbpb0]

autoencoder = Model(input_img, decoded)
のすぐ次に
print(autoencoder.summary())
を追加して実行したら、その結果の一番下の近くに
「activation_7 (Activation) (None, 128, 128, 3) 0 」
と表示されますけど、それの「(None, 128, 128, 3)」が、画像に合わせて「(None, 128, 128, 1)」とならないとダメなのではないですかね

x = Conv2D(3, (3, 3), padding='same')(x)
x = BatchNormalization()(x)
decoded = Activation('sigmoid')(x)
の「Conv2D(3,」を「Conv2D(1,」と変えたら、「(None, 128, 128, 1)」となると思います
質問者さんの意図と合ってるかは分かりませんが

[tkrd]

はい。一件目の事柄についてその通りで、reshape後はx_trainであればデータを分割しているために(117, 128, 128, 1)になります。

[tkrd]

jbpb0さんのおっしゃる通りコードを変更したところ同箇所のエラーは無事解決いたしました。ありがとうございます。。

Answer 1

x = Conv2D(3, (3, 3), padding='same')(x)
x = BatchNormalization()(x)
decoded = Activation('sigmoid')(x)
の「Conv2D(3,」を「Conv2D(1,」と変えたら、解決しました。