Yolov5学習時すべての画像ファイルが破損する

実現したいこと

yolov5で自作モデルを作りたい

前提

Roboflowを使い学習データを作成し、GoogleColabで学習を始めようと、コマンドを実行すると、
「
train: Scanning /content/drive/MyDrive/yolov5/realsense/train/labels.cache... 0 images, 0 backgrounds, 654 corrupt: 100% 654/654 [00:00<?, ?it/s]
」
というように画像がすべて破損しているらしい

解決方法というか、エラーの原因を解明してほしいです。

発生している問題・エラーメッセージ

Mounted at /content/drive
/content/drive/MyDrive/yolov5

train: weights=yolov5s.pt, cfg=yolov5m.yaml, data=/content/drive/MyDrive/yolov5/realsense/data.yaml, hyp=data/hyps/hyp.scratch-low.yaml, epochs=100, batch_size=16, imgsz=640, rect=False, resume=False, nosave=False, noval=False, noautoanchor=False, noplots=False, evolve=None, bucket=, cache=None, image_weights=False, device=, multi_scale=False, single_cls=False, optimizer=SGD, sync_bn=False, workers=8, project=runs/train, name=exp, exist_ok=False, quad=False, cos_lr=False, label_smoothing=0.0, patience=100, freeze=[0], save_period=-1, seed=0, local_rank=-1, entity=None, upload_dataset=False, bbox_interval=-1, artifact_alias=latest
Command 'git fetch origin' timed out after 5 seconds
requirements: Ultralytics requirement ['Pillow>=10.0.1'] not found, attempting AutoUpdate...
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
imageio 2.31.6 requires pillow<10.1.0,>=8.3.2, but you have pillow 10.1.0 which is incompatible.
Collecting Pillow>=10.0.1
  Downloading Pillow-10.1.0-cp310-cp310-manylinux_2_28_x86_64.whl (3.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/3.6 MB 33.9 MB/s eta 0:00:00
Installing collected packages: Pillow
  Attempting uninstall: Pillow
    Found existing installation: Pillow 9.4.0
    Uninstalling Pillow-9.4.0:
      Successfully uninstalled Pillow-9.4.0
Successfully installed Pillow-10.1.0

requirements: AutoUpdate success ✅ 5.6s, installed 1 package: ['Pillow>=10.0.1']
requirements: ⚠️ Restart runtime or rerun command for updates to take effect

YOLOv5 🚀 v7.0-240-g84ec8b5 Python-3.10.12 torch-2.1.0+cu118 CUDA:0 (Tesla T4, 15102MiB)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Transferred 57/481 items from yolov5s.pt
AMP: checks failed ❌, disabling Automatic Mixed Precision. See https://github.com/ultralytics/yolov5/issues/7908
optimizer: SGD(lr=0.01) with parameter groups 79 weight(decay=0.0), 82 weight(decay=0.0005), 82 bias
albumentations: Blur(p=0.01, blur_limit=(3, 7)), MedianBlur(p=0.01, blur_limit=(3, 7)), ToGray(p=0.01), CLAHE(p=0.01, clip_limit=(1, 4.0), tile_grid_size=(8, 8))
train: Scanning /content/drive/MyDrive/yolov5/realsense/train/labels.cache... 0 images, 0 backgrounds, 654 corrupt: 100% 654/654 [00:00<?, ?it/s]
train: WARNING ⚠️ /content/drive/MyDrive/yolov5/realsense/train/images/fruits0011eb7ed63860a5652aaafb262ff271dd64.jpg: ignoring corrupt image/label: cannot identify image file '/content/drive/MyDrive/yolov5/realsense/train/images/fruits0011eb7ed63860a5652aaafb262ff271dd64.jpg'
train: WARNING ⚠️ /content/drive/MyDrive/yolov5/realsense/train/images/fruits00225059f930072650773b4212f0bedc855a.jpg: ignoring corrupt image/label: cannot identify image file '/content/drive/MyDrive/yolov5/realsense/train/images/fruits00225059f930072650773b4212f0bedc855a.jpg'
train: WARNING ⚠️ /content/drive/MyDrive/yolov5/realsense/train/images/fruits00239d453ec8667dc30e1ac5faa2b5e69f2e.jpg: ignoring corrupt image/label: cannot identify image file '/content/drive/MyDrive/yolov5/realsense/train/images/fruits00239d453ec8667dc30e1ac5faa2b5e69f2e.jpg'
train: WARNING ⚠️ /content/drive/MyDrive/yolov5/realsense/train/images/fruits00344eb67a0ca37cd34021d4b11e21ef0d7e.jpg: ignoring corrupt image/label: cannot identify image file '/content/drive/MyDrive/yolov5/realsense/train/images/fruits00344eb67a0ca37cd34021d4b11e21ef0d7e.jpg'
train: WARNING ⚠️ /content/drive/MyDrive/yolov5/realsense/train/images/fruits00354b9a80bca97c86b0c91fd9053035735f.jpg: ignoring corrupt image/label: cannot identify image file '/content/drive/MyDrive/yolov5/realsense/train/images/fruits00354b9a80bca97c86b0c91fd9053035735f.jpg'
train: WARNING ⚠️ /content/drive/MyDrive/yolov5/realsense/train/images/fruits0046ed4bd0a8998f7ac3bcb4be8cb4a406d0.jpg: ignoring corrupt image/label: cannot identify image file '/content/drive/MyDrive/yolov5/realsense/train/images/fruits0046ed4bd0a8998f7ac3bcb4be8cb4a406d0.jpg'
train: WARNING ⚠️ /content/drive/MyDrive/yolov5/realsense/train/images/fruits00473ec3d2c028c947a382f1fc5bb2f7fa73.jpg: ignoring corrupt image/label: cannot identify image file '/content/drive/MyDrive/yolov5/realsense/train/images/fruits00473ec3d2c028c947a382f1fc5bb2f7fa73.jpg'
train: WARNING ⚠️ /content/drive/MyDrive/yolov5/realsense/train/images/fruits005839a58dfd52d9bf61505fd5fcd53ed906.jpg: ignoring corrupt image/label: cannot identify image file '/content/drive/MyDrive/yolov5/realsense/train/images/fruits005839a58dfd52d9bf61505fd5fcd53ed906.jpg'
train: WARNING ⚠️ /content/drive/MyDrive/yolov5/realsense/train/images/fruits00591f6ecc0716ba269d00aa29326f4bed4b.jpg: ignoring corrupt image/label: cannot identify image file '/content/drive/MyDrive/yolov5/realsense/train/images/fruits00591f6ecc0716ba269d00aa29326f4bed4b.jpg'
train: WARNING ⚠️ /content/drive/MyDrive/yolov5/realsense/train/images/fruits0071df9bbdbe0959b4675fd969c9d8fb61b8.jpg: ignoring corrupt image/label: cannot identify image file '/content/drive/MyDrive/yolov5/realsense/train/images/fruits0071df9bbdbe0959b4675fd969c9d8fb61b8.jpg'
train: WARNING ⚠️ /content/drive/MyDrive/yolov5/realsense/train/images/fruits00821273a24869f6bdd4951aa7fd73ac4325.jpg: ignoring corrupt image/label: cannot identify image file '/content/drive/MyDrive/yolov5/realsense/train/images/fruits00821273a24869f6bdd4951aa7fd73ac4325.jpg'
train: WARNING ⚠️ /content/drive/MyDrive/yolov5/realsense/train/images/fruits0094fb7a89a1f39e3b8ac035cfe78418ce04.jpg: ignoring corrupt image/label: cannot identify image file '/content/drive/MyDrive/yolov5/realsense/train/images/fruits0094fb7a89a1f39e3b8ac035cfe78418ce04.jpg'
train: WARNING ⚠️ /content/drive/MyDrive/yolov5/realsense/train/images/fruits009565fc51561ee78251fbe749dad0df3ae0.jpg: ignoring corrupt image/label: cannot identify image file '/content/drive/MyDrive/yolov5/realsense/train/images/fruits009565fc51561ee78251fbe749dad0df3ae0.jpg'
train: WARNING ⚠️ /content/drive/MyDrive/yolov5/realsense/train/images/fruits0107f308fe63fbf676bf6836fc3d090a4ff1.jpg: ignoring corrupt image/label: cannot identify image file '/content/drive/MyDrive/yolov5/realsense/train/images/fruits0107f308fe63fbf676bf6836fc3d090a4ff1.jpg'
train: WARNING ⚠️ /content/drive/MyDrive/yolov5/realsense/train/images/fruits011852bdd61f6f6f4f43abf2dab6fb884356.jpg: ignoring corrupt image/label: cannot identify image file '/content/drive/MyDrive/yolov5/realsense/train/images/fruits011852bdd61f6f6f4f43abf2dab6fb884356.jpg'




~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



Traceback (most recent call last):
  File "/content/drive/MyDrive/yolov5/train.py", line 647, in <module>
    main(opt)
  File "/content/drive/MyDrive/yolov5/train.py", line 536, in main
    train(opt.hyp, opt, device, callbacks)
  File "/content/drive/MyDrive/yolov5/train.py", line 195, in train
    train_loader, dataset = create_dataloader(train_path,
  File "/content/drive/MyDrive/yolov5/utils/dataloaders.py", line 124, in create_dataloader
    dataset = LoadImagesAndLabels(
  File "/content/drive/MyDrive/yolov5/utils/dataloaders.py", line 502, in __init__
    assert nf > 0 or not augment, f'{prefix}No labels found in {cache_path}, can not start training. {HELP_URL}'
AssertionError: train: No labels found in /content/drive/MyDrive/yolov5/realsense/train/labels.cache, can not start training. See https://docs.ultralytics.com/yolov5/tutorials/train_custom_data

該当のソースコード

python
1from google.colab import drive
2from google.colab import runtime
3drive.mount('/content/drive')
4%cd /content/drive/MyDrive/yolov5
5try:
6  !python train.py --cfg yolov5m.yaml --batch-size 16 --img-size 640 --data /content/drive/MyDrive/yolov5/realsense/data.yaml --weights yolov5s.pt
7
8except Exception as e:
9  runtime.unassign()

補足情報（FW/ツールのバージョンなど）

yolov5

ps_aux_grep

2023/11/11 16:28

透明度情報が付加されている https://qiita.com/tetra9064/items/965edbe65d072018b0b0 とからしいけどどうなんでしょう？

YuttyaDX

2023/11/12 02:42

試してみましたが、解決しませんでした。おそらく透明度情報は付加されていないと思われます。

hoshi-takanori

2023/11/12 05:26

imageio 2.31.6 と pillow 10.1.0 が衝突してるせいかな。pillow 10.0.1 にしてみては。 https://github.com/imageio/imageio/issues/1044

YuttyaDX

2023/11/12 14:43 編集

何度もすみません googleclablatryでimageioとimageio-ffmpegをアンインストールしてpillow10.0.1をインストールしてみたのですが、やはりエラーは変わりませんでした。

ps_aux_grep

2023/11/12 14:39

最後のAssertion Error見る感じ，カスタムデータセットの構成に誤りがあるとも取れますが，ちゃんとしたデータ構成になっていますか？

行動規範の内容に同意します

回答1件

自己解決

いままでうまくいっていたのになぜかできなくなっていました。
ps_aux_grepさんのコメントをきっかけに今までと違うところを考えていると、いつのまにかtrainフォルダに生成されていた「label.cache」があり。それを消してみたところうまく動くようになりました。
ps_aux_grepさんやhoshi-takanoriさん、ありがとうございました。

いろいろな条件で検証を行ってみたところ
透明度情報やPillowのバージョンなどは関係なく、「label.cache」によって引き起こされていたことがわかりました。
結局「label.cache」が何だったのかはわかりませんが、このページに訪れた人の参考になることを願います。

投稿2023/11/12 15:14

YuttyaDX

総合スコア25