Keras LSTM 学習のloss,val_lossが変化しない場合がある Deep Learning

Question

前提・実現したいこと

Kerasを用いてLSTMを動かしているのですが、学習がうまくいく場合とうまくいかない場合が存在するのですが、
なぜムラが存在するのかがわからないです。loss,val_lossの部分が全く変化しない時が存在します。

追記：
学習が成功している方はかなり省略していますが、どちらの場合でもloss: - mse: - val_loss: - val_mse: の値が固定されている以外は同じような表記のされ方でターミナルに書かれています。

発生している問題・エラーメッセージ

学習が成功していると思われる場合

5834/5834 [==============================] - 2s 284us/step - loss: 0.0241 - mse: 0.0241 - val_loss: 0.0049 - val_mse: 0.0049
Epoch 2/200
5834/5834 [==============================] - 1s 142us/step - loss: 0.0023 - mse: 0.0023 - val_loss: 9.0606e-04 - val_mse: 9.0606e-04








学習が上手く行っていないと考えられる場合

Using TensorFlow backend.
lstm112-2.py:50: RuntimeWarning: invalid value encountered in true_divide
  x_train = (x_train - mi)/(ma - mi)
lstm112-2.py:52: RuntimeWarning: invalid value encountered in true_divide
  y_train = (y_train - mi)/(ma - mi)
2020-03-15 00:34:13.169460: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-03-15 00:34:13.206396: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f84e82c36a0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-03-15 00:34:13.206414: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
[<tf.Tensor 'activation_1/Relu:0' shape=(None, 5, 1) dtype=float32>]
Model: "sequential_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
bidirectional_1 (Bidirection (None, 5, 256)            133120    
_________________________________________________________________
dense_1 (Dense)              (None, 5, 1)              257       
_________________________________________________________________
activation_1 (Activation)    (None, 5, 1)              0         
=================================================================
Total params: 133,377
Trainable params: 133,377
Non-trainable params: 0
_________________________________________________________________
None
Train on 5834 samples, validate on 1459 samples
Epoch 1/200
2020-03-15 00:34:14.473405: E tensorflow/core/grappler/optimizers/meta_optimizer.cc:561] remapper failed: Invalid argument: MutableGraphView::MutableGraphView error: node 'loss/activation_1_loss/mean_squared_error/weighted_loss/concat' has self cycle fanin 'loss/activation_1_loss/mean_squared_error/weighted_loss/concat'.
2020-03-15 00:34:14.486520: E tensorflow/core/grappler/optimizers/dependency_optimizer.cc:717] Iteration = 0, topological sort failed with message: The graph couldn't be sorted in topological order.
2020-03-15 00:34:14.492424: E tensorflow/core/grappler/optimizers/dependency_optimizer.cc:717] Iteration = 1, topological sort failed with message: The graph couldn't be sorted in topological order.
2020-03-15 00:34:14.549041: E tensorflow/core/grappler/optimizers/meta_optimizer.cc:561] arithmetic_optimizer failed: Invalid argument: The graph couldn't be sorted in topological order.
2020-03-15 00:34:14.552868: E tensorflow/core/grappler/optimizers/meta_optimizer.cc:561] remapper failed: Invalid argument: MutableGraphView::MutableGraphView error: node 'loss/activation_1_loss/mean_squared_error/weighted_loss/concat' has self cycle fanin 'loss/activation_1_loss/mean_squared_error/weighted_loss/concat'.
2020-03-15 00:34:14.557423: E tensorflow/core/grappler/optimizers/dependency_optimizer.cc:717] Iteration = 0, topological sort failed with message: The graph couldn't be sorted in topological order.
2020-03-15 00:34:14.561990: E tensorflow/core/grappler/optimizers/dependency_optimizer.cc:717] Iteration = 1, topological sort failed with message: The graph couldn't be sorted in topological order.
2020-03-15 00:34:14.579734: W tensorflow/core/common_runtime/process_function_library_runtime.cc:697] Ignoring multi-device function optimization failure: Invalid argument: The graph couldn't be sorted in topological order.
5600/5834 [===========================>..] - ETA: 0s - loss: 0.1171 - mse: 0.11712020-03-15 00:34:15.622048: E tensorflow/core/grappler/optimizers/meta_optimizer.cc:561] remapper failed: Invalid argument: MutableGraphView::MutableGraphView error: node 'loss/activation_1_loss/mean_squared_error/weighted_loss/concat' has self cycle fanin 'loss/activation_1_loss/mean_squared_error/weighted_loss/concat'.
2020-03-15 00:34:15.624523: E tensorflow/core/grappler/optimizers/dependency_optimizer.cc:717] Iteration = 0, topological sort failed with message: The graph couldn't be sorted in topological order.
2020-03-15 00:34:15.625641: E tensorflow/core/grappler/optimizers/dependency_optimizer.cc:717] Iteration = 1, topological sort failed with message: The graph couldn't be sorted in topological order.
2020-03-15 00:34:15.635130: E tensorflow/core/grappler/optimizers/meta_optimizer.cc:561] arithmetic_optimizer failed: Invalid argument: The graph couldn't be sorted in topological order.
2020-03-15 00:34:15.635882: E tensorflow/core/grappler/optimizers/meta_optimizer.cc:561] remapper failed: Invalid argument: MutableGraphView::MutableGraphView error: node 'loss/activation_1_loss/mean_squared_error/weighted_loss/concat' has self cycle fanin 'loss/activation_1_loss/mean_squared_error/weighted_loss/concat'.
2020-03-15 00:34:15.636640: E tensorflow/core/grappler/optimizers/dependency_optimizer.cc:717] Iteration = 0, topological sort failed with message: The graph couldn't be sorted in topological order.
2020-03-15 00:34:15.637402: E tensorflow/core/grappler/optimizers/dependency_optimizer.cc:717] Iteration = 1, topological sort failed with message: The graph couldn't be sorted in topological order.
2020-03-15 00:34:15.642129: W tensorflow/core/common_runtime/process_function_library_runtime.cc:697] Ignoring multi-device function optimization failure: Invalid argument: The graph couldn't be sorted in topological order.
5834/5834 [==============================] - 2s 279us/step - loss: 0.1172 - mse: 0.1172 - val_loss: 0.1182 - val_mse: 0.1182
Epoch 2/200
5834/5834 [==============================] - 1s 140us/step - loss: 0.1172 - mse: 0.1172 - val_loss: 0.1182 - val_mse: 0.1182

該当のソースコード

#coding:utf-8
import csv
import re
import datetime
import time
import numpy as np
import pandas as pd
import math
import pylab
import os
import os.path
import matplotlib.pyplot as plt
from keras.models import Sequential
from keras.layers.core import Dense, Activation
from keras.layers.recurrent import LSTM
from keras.layers.wrappers import Bidirectional
from keras.models import Model
from keras.optimizers import Adam
from keras.callbacks import EarlyStopping
from keras.layers import Input, LSTM, GRU, RepeatVector
from keras.layers.core import Dense, Lambda
from keras import objectives
from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn.utils import shuffle
from sklearn.utils import shuffle
from keras import initializers
from sklearn import preprocessing
from sklearn.metrics import roc_curve, auc

data = np.loadtxt("./data/main.csv",delimiter=",")
x = data[:,:5]
y = data[:,:5]
c = len(data[0]) - 1
df = pd.read_csv("./data/main.csv",delimiter=",")
length = len(df.index)
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=42)

mi = x_train.min(axis=0, keepdims=True)
ma = x_train.max(axis=0, keepdims=True)

x_train = (x_train - mi)/(ma - mi)
x_train[np.isnan(x_train)] = 0
y_train = (y_train - mi)/(ma - mi)
y_train[np.isnan(y_train)] = 0

x_train = x_train.reshape(-1, c, 1)
y_train = y_train.reshape(-1, c, 1)

x_train, x_valid, y_train, y_valid = \
    train_test_split(x_train, y_train, test_size=0.2)

'''
モデル設定
'''
n_in = 1
n_long = c
n_hidden = 128
n_out = 1

early_stopping = EarlyStopping(monitor='val_mse', patience=30, verbose=1)

model = Sequential()
model.add(Bidirectional(LSTM(n_hidden,return_sequences = True),
                        input_shape=(n_long,n_in)))
model.add(Dense(n_out, kernel_initializer=initializers.random_normal()))
model.add(Activation('relu'))

model.compile(loss='mean_squared_error',
              optimizer=Adam(lr=0.001, beta_1=0.9, beta_2=0.999),
              metrics=['mse'])
print(model.outputs)
print(model.summary())

'''
モデル学習
'''

epochs = 200
batch_size = 50

hist = model.fit(x_train, y_train,
                 batch_size=batch_size,
                 epochs=epochs,
                 validation_data=(x_valid, y_valid),
                 callbacks=[early_stopping])