OpenAi gymの実行アニメーションを保存したい

Question

前提・実現したいこと

OpenAi gymの実行アニメーションを保存したいのですが、
プログラム実行時にエラーが発生し、なにも映像が表示されないmp4ファイルが生成されてしまう。

つくりながら学ぶ！深層強化学習を参考に強化学習の勉強をしているのですが、上記の内容で躓いています。
どなたかお力を貸していただけないでしょうか。
下記は著者様の元記事
https://book.mynavi.jp/manatee/detail/id=88961

Anacondaにて環境を構築し、以下は手動にて追加したパッケージ。

gym==0.9.7
matplotlib
JSAnimation
pyglet==1.2.4
conda install -c conda-forge ffmpeg

使用言語、Python3

問題解決法をご存じであれば、ご教授のほど宜しくお願い致します。

発生している問題・エラーメッセージ

AttributeError                            Traceback (most recent call last)
<ipython-input-5-2d605f8148aa> in <module>
      1 # 動画を保存と描画
----> 2 display_frames_as_gif(frames)

<ipython-input-3-9d95926f2835> in display_frames_as_gif(frames)
     22 
     23     anim.save('movie_cartpole.mp4')  # 追記：動画の保存です
---> 24     display(display_animation(anim, default_mode='loop'))

E:\Anaconda3\envs\rl_env2\lib\site-packages\JSAnimation\IPython_display.py in display_animation(anim, **kwargs)
     84     """Display the animation with an IPython HTML object"""
     85     from IPython.display import HTML
---> 86     return HTML(anim_to_html(anim, **kwargs))
     87 
     88 

E:\Anaconda3\envs\rl_env2\lib\site-packages\JSAnimation\IPython_display.py in anim_to_html(anim, fps, embed_frames, default_mode)
     74             anim.save(f.name,  writer=HTMLWriter(fps=fps,
     75                                                  embed_frames=embed_frames,
---> 76                                                  default_mode=default_mode))
     77             html = open(f.name).read()
     78 

E:\Anaconda3\envs\rl_env2\lib\site-packages\matplotlib\animation.py in save(self, filename, writer, fps, dpi, codec, bitrate, extra_args, metadata, extra_anim, savefig_kwargs, progress_callback)
   1150                             progress_callback(frame_number, total_frames)
   1151                             frame_number += 1
-> 1152                     writer.grab_frame(**savefig_kwargs)
   1153 
   1154         # Reconnect signal for first draw if necessary

E:\Anaconda3\envs\rl_env2\lib\contextlib.py in __exit__(self, type, value, traceback)
     86         if type is None:
     87             try:
---> 88                 next(self.gen)
     89             except StopIteration:
     90                 return False

E:\Anaconda3\envs\rl_env2\lib\site-packages\matplotlib\animation.py in saving(self, fig, outfile, dpi, *args, **kwargs)
    230             yield self
    231         finally:
--> 232             self.finish()
    233 
    234 

E:\Anaconda3\envs\rl_env2\lib\site-packages\matplotlib\animation.py in finish(self)
    526         # Call run here now that all frame grabbing is done. All temp files
    527         # are available to be assembled.
--> 528         self._run()
    529         MovieWriter.finish(self)  # Will call clean-up
    530 

E:\Anaconda3\envs\rl_env2\lib\site-packages\JSAnimation\html_writer.py in _run(self)
    321             of.write(JS_INCLUDE)
    322             of.write(DISPLAY_TEMPLATE.format(id=self.new_id(),
--> 323                                              Nframes=len(self._temp_names),
    324                                              fill_frames=fill_frames,
    325                                              interval=interval,

AttributeError: 'HTMLWriter' object has no attribute '_temp_names'

該当のソースコード

# 使用するパッケージの宣言
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
import gym

# 動画の描画関数の宣言
# 参考URL http://nbviewer.jupyter.org/github/patrickmineault/xcorr-notebooks/blob/master/Render%20OpenAI%20gym%20as%20GIF.ipynb
from JSAnimation.IPython_display import display_animation
from matplotlib import animation
from IPython.display import display


def display_frames_as_gif(frames):
    """
    Displays a list of frames as a gif, with controls
    """
    plt.figure(figsize=(frames[0].shape[1]/72.0, frames[0].shape[0]/72.0),
               dpi=72)
    patch = plt.imshow(frames[0])
    plt.axis('off')

    def animate(i):
        patch.set_data(frames[i])

    anim = animation.FuncAnimation(plt.gcf(), animate, frames=len(frames),
                                   interval=50)

    anim.save('movie_cartpole.mp4')  # 追記：動画の保存です
    display(display_animation(anim, default_mode='loop'))

# CartPoleをランダムに動かす

frames = []
env = gym.make('CartPole-v0')
observation = env.reset()  # 最初に環境のresetが必要

for step in range(0, 200):
    frames.append(env.render(mode='rgb_array'))  # framesに各時刻の画像を追加していく
    action = np.random.choice(2)  # 0(カートを左に押す), 1(カートを右に押す)をランダムに返す
    observation, reward, done, info = env.step(action)  # actionを実行する

# 注意：実行後にipykernel_launcher.p...というindowが開いて固まりますが放置しておいてください

# 動画を保存と描画
display_frames_as_gif(frames)

試したこと

カーネルの更新

Answer 1

Solution 1

The first solution is a hack. You just need to make JSAnimation package compatible with the newer versions of matplotlib. And it's much easier than you might expect.

Find the JSAnimation package in your Anaconda installation. For the case of the base environment it might be:

C:\ProgramData\Anaconda3\Lib\site-packages\JSAnimation

For other environments it will be something like:

C:\ProgramData\Anaconda3\envs\your_env_name\Lib\site-packages\JSAnimation

Open html_writer.py file with Notepad++ or any other text editor, but not with PyCharm, as it will not allow you to modify a library file.

The only things you need to change are:

• Change self._temp_names to self._temp_paths.

• Change return ('', '') to return (''.encode('utf-8'), ''.encode('utf-8')) to return a tuple of byte arrays instead of strings.

Save the file (you will need admin privileges), run the Jupyter. It should work now. It worked for me.

For convenience here's code of the _run method in html_writer.py after changes:

def _run(self):
	# make a ducktyped subprocess standin
	# this is called by the MovieWriter base class, but not used here.
	class ProcessStandin(object):
		returncode = 0
		def communicate(self):
			return (''.encode('utf-8'), ''.encode('utf-8'))
	self._proc = ProcessStandin()
	# save the frames to an html file
	if self.embed_frames:
		fill_frames = _embedded_frames(self._saved_frames,
									   self.frame_format)
	else:
		# temp names is filled by FileMovieWriter
		fill_frames = _included_frames(self._temp_paths,
									   self.frame_format)
	mode_dict = dict(once_checked='',
					 loop_checked='',
					 reflect_checked='')
	mode_dict[self.default_mode + '_checked'] = 'checked'
	interval = int(1000. / self.fps)
	with open(self.outfile, 'w') as of:
		of.write(JS_INCLUDE)
		of.write(DISPLAY_TEMPLATE.format(id=self.new_id(),
										 Nframes=len(self._temp_paths),
										 fill_frames=fill_frames,
										 interval=interval,
										 icons=_Icons(),
										 **mode_dict))

Solution 2

This is a cleaner solution. As JSAnimation was merged into newer matplotlib it is better to use the integrated version. The solution is as follows:

1. Open pong_utils.py.

2. Find animate_frames method.

3. Just before this method add:

def display_animation(anim):
    plt.close(anim._fig)
    return HTML(anim.to_jshtml())

1. Change

display(display_animation(fanim, default_mode='once'))

to

display(display_animation(fanim))

1. You can remove the import:

from JSAnimation.IPython_display import display_animation

That's it.