tf.keras.application モデルがダウンロードできない

Question

tf.keras.application.Mobilenetがダウンロードできない

何が起こったか

機械学習プログラムを書こうと思い、試しに以下のコードを書きました。

from tensorflow import keras

if __name__ == "__main__":
    inputs = keras.layers.Input(shape=(128, 128, 1))
    x = keras.layers.Conv2D(filters=3, kernel_size=3, padding="same",
                            activation="relu")(inputs)
    x = keras.applications.MobileNet(include_top=False,
                                     input_shape=(128, 128, 3),
                                     weights="imagenet",
                                     pooling="max")(x)
    x = keras.layers.Dense(128, activation="relu")(x)
    x = keras.layers.Dropout(0.3)(x)
    pred = keras.layers.Dense(100, activation="softmax")(x)
    model = keras.Model(inputs=inputs, outputs=pred)
    model.summary()

mobilenetを持ってきて転移学習をしようと思ったのですがこのコードの時点で以下のエラーが発生しました。

2020-11-09 18:21:44.333555: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: c
annot open shared object file: No such file or directory
2020-11-09 18:21:44.333600: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2020-11-09 18:21:45.650596: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open
 shared object file: No such file or directory
2020-11-09 18:21:45.650656: W tensorflow/stream_executor/cuda/cuda_driver.cc:312] failed call to cuInit: UNKNOWN ERROR (303)
2020-11-09 18:21:45.650686: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (mochi-MizLab): /proc/driver/
nvidia/version does not exist              
2020-11-09 18:21:45.650988: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to use t
he following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2020-11-09 18:21:45.677190: I tensorflow/core/platform/profile_utils/cpu_utils.cc:104] CPU Frequency: 2400500000 Hz
2020-11-09 18:21:45.680459: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x563fb5dd3270 initialized for platform Host (this does not guarantee that XLA will
 be used). Devices:                        
2020-11-09 18:21:45.680515: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/mobilenet/mobilenet_1_0_128_tf_no_top.h5
Traceback (most recent call last):         
  File "/usr/lib/python3.8/urllib/request.py", line 1350, in do_open                  
    h.request(req.get_method(), req.selector, req.data, headers,                      
  File "/usr/lib/python3.8/http/client.py", line 1255, in request                     
    self._send_request(method, url, body, headers, encode_chunked)                    
  File "/usr/lib/python3.8/http/client.py", line 1301, in _send_request               
    self.endheaders(body, encode_chunked=encode_chunked)                              
  File "/usr/lib/python3.8/http/client.py", line 1250, in endheaders                  
    self._send_output(message_body, encode_chunked=encode_chunked)                    
  File "/usr/lib/python3.8/http/client.py", line 1010, in _send_output                
    self.send(msg)                         
  File "/usr/lib/python3.8/http/client.py", line 950, in send                         
    self.connect()                         
  File "/usr/lib/python3.8/http/client.py", line 1417, in connect                     
    super().connect()                                         
  File "/usr/lib/python3.8/http/client.py", line 921, in connect                                                                                                      [5/129]
    self.sock = self._create_connection(                                                                                                                                     
  File "/usr/lib/python3.8/socket.py", line 787, in create_connection                                                                                                        
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):                                                                                                                      
  File "/usr/lib/python3.8/socket.py", line 918, in getaddrinfo                                                                                                              
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):                                                                                                  
socket.gaierror: [Errno -2] Name or service not known                                 

During handling of the above exception, another exception occurred:                   

Traceback (most recent call last):         
  File "/home/mochi/.local/share/virtualenvs/machine-genome-classification-lm-emihK/lib/python3.8/site-packages/tensorflow/python/keras/utils/data_utils.py", line 278, in ge
t_file                                     
    urlretrieve(origin, fpath, dl_progress)                                           
  File "/usr/lib/python3.8/urllib/request.py", line 247, in urlretrieve               
    with contextlib.closing(urlopen(url, data)) as fp:                                
  File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen                   
    return opener.open(url, data, timeout)                                            
  File "/usr/lib/python3.8/urllib/request.py", line 525, in open                      
    response = self._open(req, data)                                                  
  File "/usr/lib/python3.8/urllib/request.py", line 542, in _open                     
    result = self._call_chain(self.handle_open, protocol, protocol +                  
  File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain               
    result = func(*args)                   
  File "/usr/lib/python3.8/urllib/request.py", line 1393, in https_open               
    return self.do_open(http.client.HTTPSConnection, req,                             
  File "/usr/lib/python3.8/urllib/request.py", line 1353, in do_open                  
    raise URLError(err)                    
urllib.error.URLError: <urlopen error [Errno -2] Name or service not known>

During handling of the above exception, another exception occurred:                   

Traceback (most recent call last):         
  File "src/machine_learning/tensorflow/foo.py", line 7, in <module>                  
    x = keras.applications.MobileNet(include_top=False,                               
  File "/home/mochi/.local/share/virtualenvs/machine-genome-classification-lm-emihK/lib/python3.8/site-packages/tensorflow/python/keras/applications/mobilenet.py", line 304,
 in MobileNet                              
    weights_path = data_utils.get_file(                                               
  File "/home/mochi/.local/share/virtualenvs/machine-genome-classification-lm-emihK/lib/python3.8/site-packages/tensorflow/python/keras/utils/data_utils.py", line 282, in ge
t_file                                     
    raise Exception(error_msg.format(origin, e.errno, e.reason))                      
Exception: URL fetch failure on https://storage.googleapis.com/tensorflow/keras-applications/mobilenet/mobilenet_1_0_128_tf_no_top.h5: None -- [Errno -2] Name or service not
known

試してみたこと

どうやらhttps://storage.googleapis.com/tensorflow/keras-applications/mobilenet/mobilenet_1_0_128_tf_no_top.h5がURLとして正しくないようなのですが、手動でこのURLにアクセスすると以下のXMLレスポンスが得られました。

<?xml version='1.0' encoding='UTF-8'?><Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Details>No such object: tensorflow/keras-applications/mobilenet/mobilenet_1_0_128_tf_no_top.h5:</Details></Error>

また、weights=Noneではモデルが読み込めたことから、

• モデルの呼び出し方法が違う
• keras内でURLをうまく組み立てられてない

のどちらかではないかと思っているのですが、どのように対処すればよいでしょうか?

ご教授よろしくお願いいたします。

環境

OS: Manjaro Linux x86_64
Python: 3.8.6
tensorflow: 2.3.1

Answer 1

いま前述のURLにアクセスしたところ、ダウンロードができました。

おそらくgoogleのサーバ側の問題だったようです。