Q&A
一般ユーザ権限でプラグインを正常にインストールできました。
有難うございます。
csvファイルのデータをembulkを使って、bigqueryにデータ連携するテストを実施しています。
データの一部に日付データがあって、カラム値にはyyyymmddの値とyyyymmddhhmmssの値が混在しています。
混在しているため、通常の方法だといづれかの値がスキップされてしまいます。
そのため、embulk-filter-timestamp_formatを使って文字列として取り込んでデータ連携させたいです。
テストを実施している中で、タイムゾーンが不正？のエラーが出力されています。embulkの定義ファイルでタイムゾーンを定義しているのですが、うまく動作しない状況となります。
■プラグイン embulk (0.9.8 java) embulk-input-oracle (0.9.3) embulk-output-bigquery (0.4.9) embulk-filter-timestamp_format (0.3.1) ■embulk定義ファイル in: type: file path_prefix: xxxxx\DATA_CSV parser: type: csv delimiter: ',' skip_header_line: false charset: UTF-8 newline: CRLF quote: '"' escape: '"' trim_if_not_quoted: false skip_header_lines: 0 allow_extra_columns: false allow_optional_columns: true columns: - {name: xxx001, type: string} - {name: xxx002, type: string} - {name: xxx003, type: timestamp} - {name: xxx004, type: timestamp} filters: - type: timestamp_format default_to_timezone: "Asia/Tokyo" default_to_timestamp_format: "%Y-%m-%d %H:%M:%S" columns: - {name: xxx003, type: timestamp, format: '%Y/%m/%d'} - {name: xxx004, type: timestamp, format: '%Y/%m/%d %H:%M:%S'} out: {type: bigquery, auth_method: json_key, json_keyfile: 'xxxxx.json', project: xxxxx, dataset: xxxx, auto_create_table: true, table: CSV_BQ_TABLE, open_read_timeout_sec: 360000, send_timeout_sec: 360000, read_timeout_sec: 360000} ■実行時のエラー Caused by: java.lang.AbstractMethodError: Method org/embulk/filter/timestamp_format/TimestampParser$TimestampParserColumnOptionImpl.getTimeZoneId()Lcom/google/common/base/Optional; is abstract at org.embulk.filter.timestamp_format.TimestampParser$TimestampParserColumnOptionImpl.getTimeZoneId(TimestampParser.java) at org.embulk.spi.time.TimestampParser.<init>(TimestampParser.java:27) at org.embulk.filter.timestamp_format.TimestampParser.getTimestampParser(TimestampParser.java:249) at org.embulk.filter.timestamp_format.TimestampParser.<init>(TimestampParser.java:70) at org.embulk.filter.timestamp_format.ColumnCaster.getTimestampParser(ColumnCaster.java:91) at org.embulk.filter.timestamp_format.ColumnCaster.buildTimestampParserMap(ColumnCaster.java:65) at org.embulk.filter.timestamp_format.ColumnCaster.<init>(ColumnCaster.java:51) at org.embulk.filter.timestamp_format.ColumnVisitorImpl.<init>(ColumnVisitorImpl.java:44) at org.embulk.filter.timestamp_format.TimestampFormatFilterPlugin$1.<init>(TimestampFormatFilterPlugin.java:162) at org.embulk.filter.timestamp_format.TimestampFormatFilterPlugin.open(TimestampFormatFilterPlugin.java:159) at org.embulk.spi.util.Filters.open(Filters.java:50) at org.embulk.spi.util.Executors.process(Executors.java:59) at org.embulk.spi.util.Executors.process(Executors.java:38) at org.embulk.exec.LocalExecutorPlugin$DirectExecutor$1.call(LocalExecutorPlugin.java:172) at org.embulk.exec.LocalExecutorPlugin$DirectExecutor$1.call(LocalExecutorPlugin.java:169) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Error: java.lang.RuntimeException: java.lang.AbstractMethodError: Method org/embulk/filter/timestamp_format/TimestampParser$TimestampParserColumnOptionImpl.getTimeZoneId()Lcom/google/common/base/Optional; is abstract
※フィルタ正常インストール後の実行結果、スキップされる。
2019-02-19 18:25:03.497 +0900: Embulk v0.9.15 2019-02-19 18:25:04.366 +0900 [WARN] (main): DEPRECATION: JRuby org.jruby.embed.ScriptingContainer is directly injected. 2019-02-19 18:25:08.661 +0900 [INFO] (main): Gem's home and path are set by default: "C:\Users\xxxx.embulk\lib\gems" 2019-02-19 18:25:10.668 +0900 [INFO] (main): Started Embulk v0.9.15 2019-02-19 18:25:18.645 +0900 [INFO] (0001:transaction): Loaded plugin embulk-output-bigquery (0.4.9) 2019-02-19 18:25:18.801 +0900 [INFO] (0001:transaction): Loaded plugin embulk-filter-timestamp_format (0.3.2) 2019-02-19 18:25:18.894 +0900 [INFO] (0001:transaction): Listing local files at directory 'C:\Users\xxxx\Desktop\embulk\DATA_CSV' filtering filename by prefix '' 2019-02-19 18:25:18.894 +0900 [INFO] (0001:transaction): "follow_symlinks" is set false. Note that symbolic links to directories are skipped. 2019-02-19 18:25:19.003 +0900 [INFO] (0001:transaction): Loading files [C:\Users\xxxx\Desktop\embulk\DATA_CSV\SRDCSBAK.B_RS_TRAN_2_11.DAT, C:\Users\xxxx\Desktop\embulk\DATA_CSV\tmp\SRDCSBAK.B_RS_TRAN_2_690.DAT] 2019-02-19 18:25:19.097 +0900 [INFO] (0001:transaction): Using local thread executor with max_threads=8 / tasks=79 2019-02-19 18:25:19.238 +0900 [INFO] (0001:transaction): embulk-output-bigquery: Get dataset... fluid-emissary-216806:embulk_test 2019-02-19 18:25:21.503 +0900 [INFO] (0001:transaction): embulk-output-bigquery: Create table... fluid-emissary-216806:embulk_test.LOAD_TEMP_a1f6a8a7_ef2d_4e0b_a061_fe5190c6ba8b_CSV_BQ_TABLE 2019-02-19 18:25:22.539 +0900 [INFO] (0001:transaction): {done: 0 / 79, running: 0}
※2019/2/20 001 定義
in: type: file path_prefix: xxx\DATA_CSV parser: type: csv delimiter: ',' skip_header_line: false charset: UTF-8 newline: CRLF quote: '"' escape: '"' trim_if_not_quoted: false skip_header_lines: 0 allow_extra_columns: false allow_optional_columns: true columns: - {name: colname001, type: string} - {name: colname002, type: string} - {name: colname003, type: string} - {name: colname004, type: timestamp} - {name: colname005, type: timestamp} - {name: colname006, type: timestamp} - {name: colname007, type: timestamp} - {name: colname008, type: timestamp} filters: - type: timestamp_format default_to_timezone: "Asia/Tokyo" default_from_timestamp_format: ["%Y/%m/%d", "%Y/%m/%d %H:%M:%S"] columns: - {name: colname004, type: timestamp, format: '%Y/%m/%d'} - {name: colname005, type: timestamp, format: '%Y/%m/%d %H:%M:%S'} - {name: colname006, type: timestamp, format: '%Y/%m/%d %H:%M:%S'} - {name: colname007, type: timestamp, format: '%Y/%m/%d %H:%M:%S'} - {name: colname008, type: timestamp, format: '%Y/%m/%d %H:%M:%S'} out: {type: bigquery, auth_method: json_key, json_keyfile: 'xxxxxx.json', project: xxxx-216806, dataset: embulk_test, auto_create_table: true, table: CSV_BQ_TABLE, open_read_timeout_sec: 360000, send_timeout_sec: 360000, read_timeout_sec: 360000}
回答7件
1
ベストアンサー
再定義し、再検証しています。（※2019/2/20 001 定義）
inの部分で定義したカラムについて、
filtersの部分で定義する必要がある。という基本的な認識で宜しいでしょうか。
はい、パーサーで入力ファイルをembulkのカラムに取り込んだのち、
フィルターで取り込んだカラムに対して処理を行います。
例ではstring4は、一度文字列で取り込んで、その後timestampに変換をしています。
（※2019/2/20 001 定義）でカラム毎にformat指定は不要だと思います。
異なるのであれば: default_from_timestamp_formatと同じ書式でかかないといけないと思います。
yaml
1in: 2 type: file 3 path_prefix: example/from_string.csv 4 parser: 5 type: csv 6 columns: 7 # 略 8 - {name: record, type: json} 9filters: 10 - type: timestamp_format 11 default_from_timezone: "Asia/Taipei" 12 default_from_timestamp_format: ["%Y-%m-%d %H:%M:%S.%N %z", "%Y-%m-%d %H:%M:%S %z", "%Y-%m-%d %H:%M:%S", "%Y-%m-%d %z", "%Y-%m-%d"] 13 default_to_timezone: "Asia/Taipei" 14 default_to_timestamp_format: "%Y-%m-%d %H:%M:%S.%N" 15 columns: 16 # 略 17 - {name: string4, type: timestamp} 18 # 略
+-----------------------------------+ | string4:string | +-----------------------------------+ | 2015-07-13 | | 2015-07-13 UTC | | 2015-07-13 00:00:00 | | 2015-07-12 16:00:00 UTC | | 2015-07-12 16:00:00.1 UTC | | 2015-07-12 16:00:00.12 UTC | | 2015-07-12 16:00:00.123 UTC | | 2015-07-12 16:00:00.1234 UTC | | 2015-07-12 16:00:00.12345 UTC | | 2015-07-12 16:00:00.123456 UTC | | 2015-07-12 16:00:00.1234567 UTC | | 2015-07-12 16:00:00.12345678 UTC | | 2015-07-12 16:00:00.123456789 UTC | +-----------------------------------+
投稿2019/02/20 00:56
編集2019/02/20 00:59
総合スコア415
1
一般権限でembulk-filter-timestamp_formatをインストールできない不具合を修正した
0.3.3をリリースしてもらいました。お試しください。
ご報告ありがとうございました。
C:\Users\user>.\embulk.bat gem install embulk-filter-timestamp_format 2019-02-20 09:32:15.411 +0900: Embulk v0.9.14 Gem plugin path is: C:\Users\user.embulk\lib\gems Fetching: embulk-filter-timestamp_format-0.3.3.gem (100%) Successfully installed embulk-filter-timestamp_format-0.3.3 1 gem installed
投稿2019/02/20 00:37
総合スコア415
回答へのコメント
1
プラグイン導入後、embulkを再実行しますと正常に動きました。
ただ、レコードがスキップされてしまう事象は改善されませんでした。
定義方法に誤りがあるのかもしれません。
embulkを使って大量データをBigQueryへバルクインサート時に一部データがスキップされる
こちらはお試しいただきましたか？
複数のタイムスタンプをパースする例をご案内したつもりなのですが、、
以下は一つしかフォーマット指定していませんね。
yaml
1 - type: timestamp_format 2 default_to_timezone: "Asia/Tokyo" 3 default_to_timestamp_format: "%Y-%m-%d %H:%M:%S"
embulk-filter-timestamp_format 0.3.2がインストールできない原因は多分わかりました。
とりあえず管理者権限で導入してください。
投稿2019/02/19 10:12
総合スコア415
回答へのコメント
ご回答有難うございます。
再定義し、再検証しています。（※2019/2/20 001 定義）
inの部分で定義したカラムについて、
filtersの部分で定義する必要がある。
という基本的な認識で宜しいでしょうか。
1
OSに導入しているjavaのバージョンが、1.8.0_192-b12になります。
最新プラグインのインストールに関係ありますでしょうか？
java 1.8.0_201で問題ありませんでした。その前は1.8.0_192-b12より古いバージョンを使っていたのでおそらく問題ないと思います。
OS Windows Server 2016
C:\Users\user\Desktop>embulk.bat gem install embulk-filter-timestamp_format 2019-02-19 17:58:36.196 +0900: Embulk v0.9.8 Gem plugin path is: C:\Users\user.embulk\lib\gems Successfully installed embulk-filter-timestamp_format-0.3.2 1 gem installed
C:\Users\user\Desktop>java -version java version "1.8.0_201" Java(TM) SE Runtime Environment (build 1.8.0_201-b09) Java HotSpot(TM) 64-Bit Server VM (build 25.201-b09, mixed mode)
投稿2019/02/19 09:00
総合スコア415
回答へのコメント
windowsコマンドプロンプトを管理者ユーザとして実行した上で
再度、インストールしたら 0.3.2バージョンをインストールできました。
プラグイン導入後、embulkを再実行しますと正常に動きました。
ただ、レコードがスキップされてしまう事象は改善されませんでした。
定義方法に誤りがあるのかもしれません。
結果は、質問内容（※フィルタ正常インストール後の実行結果、スキップされる。）に記載しております。
1
不思議ですね。こちらの環境だと発生しません。試しにEmbulk 0.9.8に変えました。
自分で試すとしたら以下のようなことを実施します。
- 他の環境で同じことをやってみる
- embulk gem uninstall embulk-filter-timestamp_format をしてから再度install
C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format*を削除して再度install
embulk gem uninstall embulk-filter-timestamp_format 2019-02-19 16:59:59.122 +0900: Embulk v0.9.8 Gem plugin path is: C:\Users\user.embulk\lib\gems Select gem to uninstall: 1. embulk-filter-timestamp_format-0.3.1 2. embulk-filter-timestamp_format-0.3.2 3. All versions > 3 Successfully uninstalled embulk-filter-timestamp_format-0.3.1 Successfully uninstalled embulk-filter-timestamp_format-0.3.2
embulk.bat gem install -V embulk-filter-timestamp_format 2019-02-19 16:58:19.878 +0900: Embulk v0.9.8 Gem plugin path is: C:\Users\user.embulk\lib\gems HEAD https://api.rubygems.org/api/v1/dependencies 200 OK GET https://api.rubygems.org/api/v1/dependencies?gems=embulk-filter-timestamp_format 200 OK Getting SRV record failed: DNS result has no information for _rubygems._tcp.api.rubygems.org C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/.gitignore C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/.travis.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/CHANGELOG.md C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/LICENSE.txt C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/README.md C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/bench/config_java.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/bench/config_jruby.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/bench/config_jruby_formatter.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/bench/config_jruby_parser.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/bench/config_nano.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/bench/gen_dummy.rb C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/build.gradle C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/classpath/JsonPathCompiler-0.1.2.jar C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/classpath/accessors-smart-1.1.jar C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/classpath/asm-5.0.3.jar C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/classpath/embulk-filter-timestamp_format-0.3.2.jar C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/classpath/json-smart-2.2.1.jar C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/classpath/slf4j-api-1.7.21.jar C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/config/checkstyle/checkstyle.xml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/bracket_notation.txt C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/bracket_notation.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/empty.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/example.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_double.csv C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_double.txt C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_double.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_long.csv C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_long.txt C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_long.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_string.csv C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_string.txt C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_string.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_string_auto_java.txt C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_string_auto_java.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_string_java.txt C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_string_java.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_timestamp.csv C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_timestamp.txt C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_timestamp.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/nested.jsonl C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/nested.txt C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/nested.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/timezone.csv C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/timezone.txt C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/timezone.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/gradle/wrapper/gradle-wrapper.jar C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/gradle/wrapper/gradle-wrapper.properties C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/gradlew C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/gradlew.bat C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/lib/embulk/filter/timestamp_format.rb C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/settings.gradle C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/ColumnCaster.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/ColumnVisitorImpl.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/JsonCaster.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/JsonPathUtil.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/JsonVisitor.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/TimestampFormatConverter.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/TimestampFormatFilterPlugin.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/TimestampFormatter.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/TimestampParser.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/TimestampUnit.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/TimestampUnitDeserializer.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/cast/DoubleCast.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/cast/LongCast.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/cast/StringCast.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/cast/TimestampCast.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/test/java/org/embulk/filter/timestamp_format/TestTimestampFormatConverter.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/test/java/org/embulk/filter/timestamp_format/TestTimestampParser.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/test/java/org/embulk/filter/timestamp_format/TestTimestampUnit.java Successfully installed embulk-filter-timestamp_format-0.3.2 1 gem installed
投稿2019/02/19 08:03
総合スコア415
回答へのコメント
アンインストール、再インストールを実行しても結果は変わらずでした。。
embulk-0.9.8でも、embulk-0.9.15でも変わらずです。
embulk-filter-timestamp_format-0.3.1でしたら、何度実施しても正常にインストール出来ます。
embulk-filter-timestamp_format-0.3.1では、潜在バグがあって、使用できない。という認識で宜しいでしょうか。
はい、タイムスタンプ周りのエラーは0.3.1までにあった不具合で0.3.2で直っているものだと思います。
OSに導入しているjavaのバージョンが、1.8.0_192-b12になります。
最新プラグインのインストールに関係ありますでしょうか？
1
Windows Server 2016だと大丈夫のようです。
Gem plugin path is表示されるパスの権限がなにか問題があったりしますか？
-Vをつけてembulk.bat gem install -V embulk-filter-timestamp_formatとしたらどうなりますか？C:\Users\user.embulk(パスは適宜変更してください)をリネームして、bigqueryとoracle、timestamp_formatを再インストールできますか？- embulk 0.9.14にしても一緒でしょうか？
C:\Users\user\Desktop>embulk.bat gem install embulk-filter-timestamp_format -v 0.3.1 2019-02-19 16:26:43.425 +0900: Embulk v0.9.14 Gem plugin path is: C:\Users\user.embulk\lib\gems Successfully installed embulk-filter-timestamp_format-0.3.1 1 gem installed C:\Users\user\Desktop>embulk.bat gem install embulk-filter-timestamp_format 2019-02-19 16:27:27.606 +0900: Embulk v0.9.14 Gem plugin path is: C:\Users\user.embulk\lib\gems Successfully installed embulk-filter-timestamp_format-0.3.2 1 gem installed
-Vの出力例
C:\Users\user\Desktop>embulk.bat gem install -V embulk-filter-timestamp_format 2019-02-19 16:30:37.247 +0900: Embulk v0.9.14 Gem plugin path is: C:\Users\user.embulk\lib\gems HEAD https://api.rubygems.org/api/v1/dependencies 200 OK GET https://api.rubygems.org/api/v1/dependencies?gems=embulk-filter-timestamp_format 200 OK Getting SRV record failed: DNS result has no information for _rubygems._tcp.api.rubygems.org C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/.gitignore C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/.travis.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/CHANGELOG.md C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/LICENSE.txt C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/README.md C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/bench/config_java.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/bench/config_jruby.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/bench/config_jruby_formatter.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/bench/config_jruby_parser.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/bench/config_nano.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/bench/gen_dummy.rb C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/build.gradle C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/classpath/JsonPathCompiler-0.1.2.jar C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/classpath/accessors-smart-1.1.jar C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/classpath/asm-5.0.3.jar C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/classpath/embulk-filter-timestamp_format-0.3.2.jar C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/classpath/json-smart-2.2.1.jar C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/classpath/slf4j-api-1.7.21.jar C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/config/checkstyle/checkstyle.xml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/bracket_notation.txt C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/bracket_notation.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/empty.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/example.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_double.csv C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_double.txt C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_double.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_long.csv C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_long.txt C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_long.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_string.csv C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_string.txt C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_string.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_string_auto_java.txt C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_string_auto_java.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_string_java.txt C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_string_java.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_timestamp.csv C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_timestamp.txt C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/from_timestamp.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/nested.jsonl C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/nested.txt C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/nested.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/timezone.csv C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/timezone.txt C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/example/timezone.yml C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/gradle/wrapper/gradle-wrapper.jar C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/gradle/wrapper/gradle-wrapper.properties C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/gradlew C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/gradlew.bat C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/lib/embulk/filter/timestamp_format.rb C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/settings.gradle C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/ColumnCaster.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/ColumnVisitorImpl.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/JsonCaster.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/JsonPathUtil.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/JsonVisitor.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/TimestampFormatConverter.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/TimestampFormatFilterPlugin.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/TimestampFormatter.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/TimestampParser.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/TimestampUnit.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/TimestampUnitDeserializer.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/cast/DoubleCast.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/cast/LongCast.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/cast/StringCast.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/main/java/org/embulk/filter/timestamp_format/cast/TimestampCast.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/test/java/org/embulk/filter/timestamp_format/TestTimestampFormatConverter.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/test/java/org/embulk/filter/timestamp_format/TestTimestampParser.java C:/Users/user/.embulk/lib/gems/gems/embulk-filter-timestamp_format-0.3.2/src/test/java/org/embulk/filter/timestamp_format/TestTimestampUnit.java Successfully installed embulk-filter-timestamp_format-0.3.2 1 gem installed
投稿2019/02/19 07:33
総合スコア415
回答へのコメント
ご回答有難うございます。
バージョン変更して、再トライした結果を質問内容に追記させて頂きました。やはり同じエラーで失敗してしまいます。
oracle、embulkの再インストールは難しい状況となります。
1
embulk-filter-timestamp_format 0.3.2を使ってみてください。
0.3.1で発生していた不具合のように思えます。
もし何らかの事情でダウンロードできない場合は
https://rubygems.org/gems/embulk-filter-timestamp_format/versions/0.3.2
のDownloadを押してgemファイルをダウンロードしたのち以下のようにすればいいと思います。
embulk gem install -l embulk-filter-timestamp_format-0.3.2.gem
投稿2019/02/19 06:58
総合スコア415
回答へのコメント
ご回答有難うございます。
0.3.2.gemファイルをインストールしますと、以下のエラーが出力されてしまいます。
ERROR: While executing gem... (Errrno::EACCES)
Permission denied - File.symlink
0.3.1バージョンのアンインストールが必要になるのでしょうか？
どうやってインストールされましたか？
OSはなんですか？
以下もためしてみていただけますか？
embulk gem install embulk-filter-timestamp_format -v 0.3.2
環境の説明しておりませんでした。すみません。
環境：Windows 7 Enterprise、Embulk 0.9.8
インストールコマンド：
(1)embulk gem install -l embulk-filter-timestamp_format-0.3.2.gem
(2)embulk gem install embulk-filter-timestamp_format -v 0.3.2
(1)、(2)、どちらのコマンドでも失敗致しました。
エラー
ERROR: While executing gem... (Errrno::EACCES)
Permission denied - File.symlink
あなたの回答
tips
プレビュー
下記のような回答は推奨されていません。
このような回答には修正を依頼しましょう。
また依頼した内容が修正された場合は、修正依頼を取り消すようにしましょう。
2019/02/20 01:22
2019/02/20 01:29
2019/02/20 01:57