Keras-rlの公式exampleでのエラー

前提・実現したいこと

強化学習を試したく、keras-rlをインストールして公式の下記exampleを実行しました。
https://github.com/keras-rl/keras-rl/blob/master/examples/ddpg_pendulum.py

しかしエラーが出てきて困っています。
アドバイスの程よろしくお願いいたします。
環境はWin10 / JupyterLab
以下の表示が出た後に問題のエラーが出ます。

==================================================================================================
Total params: 2,305
Trainable params: 2,305
Non-trainable params: 0

None

発生している問題・エラーメッセージ

---------------------------------------------------------------------------

TypeError                                 Traceback (most recent call last)

<ipython-input-3-49ce8198fa31> in <module>()
     55 agent = DDPGAgent(nb_actions=nb_actions, actor=actor, critic=critic, critic_action_input=action_input,
     56                   memory=memory, nb_steps_warmup_critic=100, nb_steps_warmup_actor=100,
---> 57                   random_process=random_process, gamma=.99, target_model_update=1e-3)
     58 agent.compile(Adam(lr=.001, clipnorm=1.), metrics=['mae'])
     59 

1 frames

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/keras_tensor.py in __len__(self)
    238 
    239   def __len__(self):
--> 240     raise TypeError('Keras symbolic inputs/outputs do not '
    241                     'implement `__len__`. You may be '
    242                     'trying to pass Keras symbolic inputs/outputs '

TypeError: Keras symbolic inputs/outputs do not implement `__len__`. You may be trying to pass Keras symbolic inputs/outputs to a TF API that does not register dispatching, preventing Keras from automatically converting the API call to a lambda layer in the Functional Model. This error will also get raised if you try asserting a symbolic input/output directly.

該当のソースコード

python

import numpy as np
import gym

from keras.models import Sequential, Model
from keras.layers import Dense, Activation, Flatten, Input, Concatenate
from keras.optimizers import Adam

from rl.agents import DDPGAgent
from rl.memory import SequentialMemory
from rl.random import OrnsteinUhlenbeckProcess


ENV_NAME = 'Pendulum-v0'


# Get the environment and extract the number of actions.
env = gym.make(ENV_NAME)
np.random.seed(123)
env.seed(123)
assert len(env.action_space.shape) == 1
nb_actions = env.action_space.shape[0]

# Next, we build a very simple model.
actor = Sequential()
actor.add(Flatten(input_shape=(1,) + env.observation_space.shape))
actor.add(Dense(16))
actor.add(Activation('relu'))
actor.add(Dense(16))
actor.add(Activation('relu'))
actor.add(Dense(16))
actor.add(Activation('relu'))
actor.add(Dense(nb_actions))
actor.add(Activation('linear'))
print(actor.summary())

action_input = Input(shape=(nb_actions,), name='action_input')
observation_input = Input(shape=(1,) + env.observation_space.shape, name='observation_input')
flattened_observation = Flatten()(observation_input)
x = Concatenate()([action_input, flattened_observation])
x = Dense(32)(x)
x = Activation('relu')(x)
x = Dense(32)(x)
x = Activation('relu')(x)
x = Dense(32)(x)
x = Activation('relu')(x)
x = Dense(1)(x)
x = Activation('linear')(x)
critic = Model(inputs=[action_input, observation_input], outputs=x)
print(critic.summary())

# Finally, we configure and compile our agent. You can use every built-in Keras optimizer and
# even the metrics!
memory = SequentialMemory(limit=100000, window_length=1)
random_process = OrnsteinUhlenbeckProcess(size=nb_actions, theta=.15, mu=0., sigma=.3)
agent = DDPGAgent(nb_actions=nb_actions, actor=actor, critic=critic, critic_action_input=action_input,
                  memory=memory, nb_steps_warmup_critic=100, nb_steps_warmup_actor=100,
                  random_process=random_process, gamma=.99, target_model_update=1e-3)
agent.compile(Adam(lr=.001, clipnorm=1.), metrics=['mae'])

# Okay, now it's time to learn something! We visualize the training here for show, but this
# slows down training quite a lot. You can always safely abort the training prematurely using
# Ctrl + C.
agent.fit(env, nb_steps=50000, visualize=True, verbose=1, nb_max_episode_steps=200)

# After training is done, we save the final weights.
agent.save_weights('ddpg_{}_weights.h5f'.format(ENV_NAME), overwrite=True)

# Finally, evaluate our algorithm for 5 episodes.
agent.test(env, nb_episodes=5, visualize=True, nb_max_episode_steps=200)