Q&A
使おうとしているFPGAのCPUコアのアーキは何ですか？
前提・実現したいこと
blueoilについての質問です。
blueoilの作業の中で、学習したデータをFPGA用にconvertする段階の作業をしています。
convertを実行すると、FPGAでの推論に必要なlibdlk_fpga.soファイルだけが出力されず、困っています。
長くなりますが、実行した時の出力を載せます。文字数の問題でだいぶ省略してあります。
発生している問題・エラーメッセージ
(.venv) myname@wood:~/blueoil$ sudo docker run --rm -e CUDA_VISIBLE_DEVICES=-1 -e OUTPUT_DIR=/home/blueoil/saved -v $(pwd)/saved:/home/blueoil/saved blueoil_root:v0.30.0-7-g0c9160b blueoil convert -e objectdetection_20210802054422 WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/horovod/tensorflow/__init__.py:163: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead. WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/horovod/tensorflow/__init__.py:189: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead. WARNING:tensorflow:From /home/blueoil/blueoil/layers/layers.py:67: conv2d (from tensorflow.python.layers.convolutional) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.keras.layers.Conv2D` instead. WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/layers/convolutional.py:424: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version. Instructions for updating: Please use `layer.__call__` method instead. WARNING:tensorflow: The TensorFlow contrib module will not be included in TensorFlow 2.0. WARNING:tensorflow:From /home/blueoil/blueoil/quantizations/binary.py:90: The name tf.where_v2 is deprecated. Please use tf.compat.v2.where instead. WARNING:tensorflow:From /home/blueoil/blueoil/networks/object_detection/yolo_v2.py:693: The name tf.space_to_depth is deprecated. Please use tf.compat.v1.space_to_depth instead. 2021-08-11 07:34:52.492574: E tensorflow/stream_executor/cuda/cuda_driver.cc:318] failed call to cuInit: UNKNOWN ERROR (303) WARNING:tensorflow:From /home/blueoil/blueoil/utils/executor.py:94: convert_variables_to_constants (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.compat.v1.graph_util.convert_variables_to_constants` WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/graph_util_impl.py:277: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.compat.v1.graph_util.extract_sub_graph` ----------------- config value -------------------- {'BATCH_SIZE': 1, 'CLASSES': ['Humanface'], 'DATASET': {'AUGMENTOR': [Brightness({'min_value': 0.75, 'max_value': 1.25}), Color({'min_value': 0.75, 'max_value': 1.25}), FlipLeftRight({'probability': 0.5}), Hue({'min_value': -10, 'max_value': 10}), SSDRandomCrop({'sample_options': (None, (0.1, None), (0.3, None), (0.7, None), (0.9, None), (None, None)), 'aspect_ratio_min': 0.5, 'aspect_ratio_max': 2, 'min_crop_ratio': 0.3})], 'BATCH_SIZE': 1, 'DATA_FORMAT': 'NHWC', 'ENABLE_PREFETCH': True, 'PRE_PROCESSOR': [ResizeWithGtBoxes({'size': [224, 224], 'resample': 'NEAREST'}), PerImageStandardization({})]}, 'DATASET_CLASS': <class 'abc.DATASET_CLASS'>, 'DATA_FORMAT': 'NHWC', 'IMAGE_SIZE': [224, 224], 'IS_DEBUG': False, 'IS_PRETRAIN': False, 'KEEP_CHECKPOINT_MAX': 5, 'MAX_EPOCHS': 1000, 'NETWORK': {'ACTIVATION_QUANTIZER': <function linear_mid_tread_half_quantizer at 0x7fbacbbc27b8>, 'ACTIVATION_QUANTIZER_KWARGS': {'bit': 2, 'max_value': 2.0}, 'ANCHORS': [(1.3221, 1.73145), (3.19275, 4.00944), (5.05587, 8.09892), (9.47112, 4.84053), (11.2364, 10.0071)], 'BATCH_SIZE': 1, 'CLASS_SCALE': 1.0, 'COORDINATE_SCALE': 1.0, 'DATA_FORMAT': 'NHWC', 'IMAGE_SIZE': [224, 224], 'LEARNING_RATE_FUNC': <function piecewise_constant at 0x7fbad7725158>, 'LEARNING_RATE_KWARGS': {'boundaries': [53675, 107350, 161026], 'values': [0.001, 0.0001, 1e-05, 1e-06]}, 'LOSS_IOU_THRESHOLD': 0.6, 'LOSS_WARMUP_STEPS': 500, 'NMS_IOU_THRESHOLD': 0.5, 'NMS_MAX_OUTPUT_SIZE': 100, 'NO_OBJECT_SCALE': 1.0, 'OBJECT_SCALE': 5.0, 'OPTIMIZER_CLASS': <class 'tensorflow.python.training.adam.AdamOptimizer'>, 'OPTIMIZER_KWARGS': {}, 'QUANTIZE_FIRST_CONVOLUTION': False, 'QUANTIZE_LAST_CONVOLUTION': False, 'SCORE_THRESHOLD': 0.05, 'WEIGHT_DECAY_RATE': 0.0005, 'WEIGHT_QUANTIZER': <function binary_channel_wise_mean_scaling_quantizer at 0x7fbacbbc2620>, 'WEIGHT_QUANTIZER_KWARGS': {}}, 'NETWORK_CLASS': <class 'blueoil.networks.object_detection.lm_fyolo.LMFYoloQuantize'>, 'NMS': <class 'blueoil.post_processor.NMS'>, 'POST_PROCESSOR': [FormatYoloV2({'image_size': [224, 224], 'num_classes': 1, 'anchors': [(1.3221, 1.73145), (3.19275, 4.00944), (5.05587, 8.09892), (9.47112, 4.84053), (11.2364, 10.0071)], 'boxes_per_cell': 5, 'data_format': 'NHWC'}), ExcludeLowScoreBox({'threshold': 0.05}), NMS({'classes': ['Humanface'], 'iou_threshold': 0.5, 'max_output_size': 100, 'per_class': True})], 'PRETRAIN_DIR': '', 'PRETRAIN_FILE': '', 'PRETRAIN_VARS': [], 'PRE_PROCESSOR': [ResizeWithGtBoxes({'size': [224, 224], 'resample': 'NEAREST'}), PerImageStandardization({})], 'SAVE_CHECKPOINT_STEPS': 1000, 'SUMMARISE_STEPS': 100, 'TASK': <Tasks.OBJECT_DETECTION: 'IMAGE.OBJECT_DETECTION'>, 'TEST_STEPS': 1000} ----------------- config value -------------------- 省略 finish import pb file optimize graph step: start optimize graph step: done! generate code step: start generate code step: done! Building lm_x86.elf Building libdlk_x86.so Building libdlk_x86.a Building lm_x86_avx.elf Building libdlk_x86_avx.so Building libdlk_x86_avx.a Building lm_arm.elf Building libdlk_arm.so Building libdlk_arm.a In file included from /usr/arm-linux-gnueabihf/include/c++/8/vector:69, from ./include/time_measurement.h:21, from src/time_measurement.cpp:18: /usr/arm-linux-gnueabihf/include/c++/8/bits/vector.tcc: In member function ‘void std::vector<_Tp, _Alloc>::_M_realloc_insert(std::vector<_Tp, _Alloc>::iterator, _Args&& ...) [with _Args = {Measurement::measure&}; _Tp = Measurement::measure; _Alloc = std::allocator<Measurement::measure>]’: /usr/arm-linux-gnueabihf/include/c++/8/bits/vector.tcc:413:7: note: parameter passing for argument of type ‘std::vector<Measurement::measure>::iterator’ {aka ‘__gnu_cxx::__normal_iterator<Measurement::measure*, std::vector<Measurement::measure> >’} changed in GCC 7.1 vector<_Tp, _Alloc>:: ^~~~~~~~~~~~~~~~~~~ /usr/arm-linux-gnueabihf/include/c++/8/bits/vector.tcc: In member function ‘void std::vector<_Tp, _Alloc>::_M_realloc_insert(std::vector<_Tp, _Alloc>::iterator, _Args&& ...) [with _Args = {const Measurement::measure&}; _Tp = Measurement::measure; _Alloc = std::allocator<Measurement::measure>]’: /usr/arm-linux-gnueabihf/include/c++/8/bits/vector.tcc:413:7: note: parameter passing for argument of type ‘std::vector<Measurement::measure>::iterator’ {aka ‘__gnu_cxx::__normal_iterator<Measurement::measure*, std::vector<Measurement::measure> >’} changed in GCC 7.1 /usr/arm-linux-gnueabihf/include/c++/8/bits/vector.tcc: In static member function ‘static void Measurement::Start(const string&)’: /usr/arm-linux-gnueabihf/include/c++/8/bits/vector.tcc:109:4: note: parameter passing for argument of type ‘__gnu_cxx::__normal_iterator<Measurement::measure*, std::vector<Measurement::measure> >’ changed in GCC 7.1 _M_realloc_insert(end(), std::forward<_Args>(__args)...); ^~~~~~~~~~~~~~~~~ /usr/arm-linux-gnueabihf/include/c++/8/bits/vector.tcc:109:4: note: parameter passing for argument of type ‘__gnu_cxx::__normal_iterator<Measurement::measure*, std::vector<Measurement::measure> >’ changed in GCC 7.1 _M_realloc_insert(end(), std::forward<_Args>(__args)...); ^~~~~~~~~~~~~~~~~ In file included from /usr/arm-linux-gnueabihf/include/c++/8/vector:64, from ./include/time_measurement.h:21, from src/time_measurement.cpp:18: /usr/arm-linux-gnueabihf/include/c++/8/bits/stl_vector.h:1085:4: note: parameter passing for argument of type ‘__gnu_cxx::__normal_iterator<Measurement::measure*, std::vector<Measurement::measure> >’ changed in GCC 7.1 _M_realloc_insert(end(), __x); ^~~~~~~~~~~~~~~~~ Building lm_arm_fpga.elf Building libdlk_arm_fpga.so Building libdlk_arm_fpga.a Building lm_aarch64.elf Building libdlk_aarch64.so Building libdlk_aarch64.a Building lm_aarch64_fpga.elf Building libdlk_aarch64_fpga.so Building libdlk_aarch64_fpga.a Output files are generated in /home/blueoil/saved/objectdetection_20210802054422/export/save.ckpt-161187/224x224/output Please see /home/blueoil/saved/objectdetection_20210802054422/export/save.ckpt-161187/224x224/output/README.md to run prediction
該当のソースコード
試したこと
blueoilには別の方法で同じようなconvertを行うやり方もありますが、そちらでもlibdlk_fpga.soだけが出力されません。
自分の考えでは、その変換のステップに必要なライブラリやパッケージがインストールされてないせいだと思いますが、それらしいエラーが出ていないのでよくわかりません。
作業の中でtensorflowを使っているようなのでそれ関連の問題かもしれません。
補足情報（FW/ツールのバージョンなど）
OS : ubuntu 20.04.2 LTS
Python : Python3.6.14
tensorflow-gpu : 1.15.2
Docker : 20.10.7
回答2件
0
自己解決
convert時にいろいろなファイルが生成されますが、その中のlbdlk_arm_fpga.soを代用することでFPGAにて実行することができました。
私の環境ではlibdlk_fpga.soは生成されないようです。
blueoil自体が数年前の環境でのプロジェクトだったのでこういうことが起きたのでしょう。
中身を理解つつやらないといけなかったことが今回の反省点です。
ありがとうございました。
投稿2021/08/17 04:13
総合スコア0
0
5.9. Shared librariesには、色々書いてありますが、
shell
1make clean 2make build ARCH=fpga USE_FPGA=enable TYPE=dynamic
というのをやっていますか？
投稿2021/08/12 10:12
総合スコア24666
はい。それをすると確かにlibdlk_fpga.soは生成されるのですが、それを使ってFPGAで動かそうとすると、
OSError: ../models/lib/libdlk_fpga.so: undefined symbol: _ZN3dlk4impl21QuantizedConv2DKn2RowERK10TensorViewI15QuantizedPackedIjEL12MemoryLayout13EERKS1_IS3_LS4_11EERK29binary_convolution_parameters
となってしまい、解決策がわからないのであきらめてしまいました。
あなたの回答
tips
プレビュー
