Q&A
csvファイルのデータをOracleDBへバルクインサートを試みると失敗します。
csvファイルは16万行、81列のファイルになります。
環境：windows7 , oracle 12c , embulk 0.9.8
※検証用の業務データのため、エラー内容から出力されたデータ項目は除いています。
embulk定義ファイル
インプット in: type: file path_prefix: 'C:\Users\ｘｘｘ\Desktop\embulk\ｘｘｘｘ.csv' parser: type: csv delimiter: ',' skip_header_line: false stop_on_invalid_record: true out: type: oracle host: ｘｘｘｘ port: ｘｘｘ database: ｘｘｘ user: ｘｘｘ password: ｘｘｘ table: ｘｘｘ mode: insert insert_method: normal driver_path: 'C:\Users\ｘｘｘ\Desktop\embulkDB\product\12.2.0\dbhome_1\jdbc\lib\ojdbc8.jar' アウトプット in: type: file path_prefix: C:\Users\ｘｘｘ\Desktop\embulk\ｘｘｘｘ.csv parser: type: csv delimiter: ',' skip_header_line: false stop_on_invalid_record: true charset: UTF-8 newline: CRLF quote: '"' escape: '"' trim_if_not_quoted: false skip_header_lines: 1 allow_extra_columns: false allow_optional_columns: false columns: - {name: ORDER_NO, type: long} - {name: RS_TYPE, type: long} - {name: FACTORY_CODE, type: long} - {name: ORG_CODE, type: string} - {name: TRAN_DATE, type: timestamp, format: '%Y/%m/%d'} - {name: HINBAN, type: string} - {name: QTY, type: long} - {name: SLIP_NO, type: string} - {name: ITEM_NO, type: string} - {name: DIVISION_NO, type: long} - {name: FROM_FACTORY_CODE, type: long} - {name: FROM_ORG_CODE, type: string} - {name: FROM_SUBINV_CODE, type: string} - {name: FROM_VALUE_CODE, type: string} - {name: FROM_OPERATOR_NAME, type: string} - {name: TO_FACTORY_CODE, type: string} - {name: TO_ORG_CODE, type: string} - {name: TO_SUBINV_CODE, type: long} - {name: TO_VALUE_CODE, type: long} - {name: TO_OPERATOR_NAME, type: string} - {name: SUB_SYSTEM_TYPE1, type: string} - {name: CANCEL_FLAG, type: long} - {name: ACT_CREATION_DATE, type: timestamp, format: '%Y/%m/%d %H:%M:%S'} - {name: WH_ACT_CREATION_DATE, type: timestamp, format: '%Y/%m/%d %H:%M:%S'} - {name: JOB_CODE, type: string} - {name: UNIT_PRICE, type: string} - {name: ERROR_STATUS, type: long} - {name: DESCRIPTION, type: string} - {name: LOCATION, type: string} - {name: REVISION_NO, type: string} - {name: OPE_CODE, type: string} - {name: BOX_KIND, type: string} - {name: DEMAND_NO, type: string} - {name: SUB_NO, type: string} - {name: TRAFFIC_NO, type: string} - {name: IF_FLAG, type: long} - {name: SELL_BUY_DATA_FLAG, type: long} - {name: SOURCE_ORDER_NO, type: long} - {name: FT_DEFINE_ID, type: long} - {name: FROM_SELL_BUY_ID, type: string} - {name: TO_SELL_BUY_ID, type: string} - {name: CREATION_DATE, type: timestamp, format: '%Y/%m/%d %H:%M:%S'} - {name: CREATED_BY, type: string} - {name: LAST_UPDATE_DATE, type: timestamp, format: '%Y/%m/%d %H:%M:%S'} - {name: LAST_UPDATED_BY, type: string} - {name: UPDATE_REQUEST_ID, type: long} - {name: TIME_STAMP, type: timestamp, format: '%Y/%m/%d %H:%M:%S'} - {name: INV_UPDATE_FLAG, type: long} - {name: ISSUE_SEQ, type: string} - {name: SOUMEI_ID, type: long} - {name: GCPCS_IF_FLAG, type: string} - {name: GCPCS_IF_CREATION_DATE, type: string} - {name: RECIEPT_CODE, type: string} - {name: DELIV_BIN_NO, type: string} - {name: DELIV_NO, type: string} - {name: RECV_BIN_NO, type: string} - {name: SUB_SYSTEM_TYPE2, type: string} - {name: SHIP_DATE, type: timestamp, format: '%Y/%m/%d'} - {name: C_NO, type: long} - {name: T_NO, type: long} - {name: RECV_OFFICE_CODE, type: long} - {name: SR_IF_FLAG, type: long} - {name: SR_IF_CREATION_DATE, type: timestamp, format: '%Y/%m/%d %H:%M:%S'} - {name: PS_IF_FLAG, type: long} - {name: PS_IF_CREATION_DATE, type: timestamp, format: '%Y/%m/%d %H:%M:%S'} - {name: CUSTOMER_CODE, type: long} - {name: KS_TYPE, type: string} - {name: SUPPLY_SECTION, type: string} - {name: MANAGE_NO, type: string} - {name: SEND_OFFICE_CODE, type: long} - {name: SOUMEI_SLIP_NO, type: long} - {name: SOUMEI_ITEM_NO, type: string} - {name: PS_YAZAKI_HINBAN, type: long} - {name: INVOICE_CASE_INFO_NO, type: string} - {name: GYK_SITE_CODE, type: string} - {name: CASE_NO, type: string} - {name: CASE_SUB_NO, type: string} - {name: FROM_LOCATION, type: string} - {name: TO_LOCATION, type: string} - {name: GYK_TRAN_TYPE, type: string} - {name: INV_INFO_UPDATE_FLAG, type: string} out: {type: oracle, host: ｘｘｘｘ, port: ｘｘｘ, database: ｘｘｘ, user: ｘｘｘ, password: ｘｘｘ, table: ｘｘｘｘ, mode: insert, insert_method: normal, driver_path: 'C:\Users\ｘｘｘ\Desktop\embulkDB\product\12.2.0\dbhome_1\jdbc\lib\ojdbc8.jar'}
※2018/11/26 現在のエラー
org.embulk.exec.PartialExecutionException: org.jruby.exceptions.RaiseException: (Error) failed during waiting a Copy job, get_job(fluid-emissary-216806, embulk_copy_job_f19fafe0-795f-4b9d-a49d-35269a460a2f), errors:[{:reason=>"invalid", :message=>"Provided Schema does not match Table fluid-emissary-216806:embulk_test.T_RS_TRAN. Field ORDER_NO has changed type from FLOAT to INTEGER."}] at org.embulk.exec.BulkLoader$LoaderState.buildPartialExecuteException(BulkLoader.java:339) at org.embulk.exec.BulkLoader.doRun(BulkLoader.java:565) at org.embulk.exec.BulkLoader.access$000(BulkLoader.java:34) at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:352) at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:349) at org.embulk.spi.Exec.doWith(Exec.java:22) at org.embulk.exec.BulkLoader.run(BulkLoader.java:349) at org.embulk.EmbulkEmbed.run(EmbulkEmbed.java:161) at org.embulk.EmbulkRunner.runInternal(EmbulkRunner.java:292) at org.embulk.EmbulkRunner.run(EmbulkRunner.java:156) at org.embulk.cli.EmbulkRun.runSubcommand(EmbulkRun.java:436) at org.embulk.cli.EmbulkRun.run(EmbulkRun.java:91) at org.embulk.cli.Main.main(Main.java:26) Caused by: org.jruby.exceptions.RaiseException: (Error) failed during waiting a Copy job, get_job(fluid-emissary-216806, embulk_copy_job_f19fafe0-795f-4b9d-a49d-35269a460a2f), errors:[{:reason=>"invalid", :message=>"Provided Schema does not match Table fluid-emissary-216806:embulk_test.T_RS_TRAN. Field ORDER_NO has changed type from FLOAT to INTEGER."}] at RUBY.wait_load(C:/Users/yazaki/.embulk/lib/gems/gems/embulk-output-bigquery-0.4.9/lib/embulk/output/bigquery/bigquery_client.rb:346) at RUBY.block in copy(C:/Users/yazaki/.embulk/lib/gems/gems/embulk-output-bigquery-0.4.9/lib/embulk/output/bigquery/bigquery_client.rb:289) at RUBY.with_job_retry(C:/Users/yazaki/.embulk/lib/gems/gems/embulk-output-bigquery-0.4.9/lib/embulk/output/bigquery/bigquery_client.rb:58) at RUBY.copy(C:/Users/yazaki/.embulk/lib/gems/gems/embulk-output-bigquery-0.4.9/lib/embulk/output/bigquery/bigquery_client.rb:249) at RUBY.transaction(C:/Users/yazaki/.embulk/lib/gems/gems/embulk-output-bigquery-0.4.9/lib/embulk/output/bigquery.rb:407) at RUBY.transaction(uri:classloader:/gems/embulk-0.9.8-java/lib/embulk/output_plugin.rb:64) Error: org.jruby.exceptions.RaiseException: (Error) failed during waiting a Copy job, get_job(fluid-emissary-216806, embulk_copy_job_f19fafe0-795f-4b9d-a49d-35269a460a2f), errors:[{:reason=>"invalid", :message=>"Provided Schema does not match Table fluid-emissary-216806:embulk_test.T_RS_TRAN. FieldORDER_NO has changed type from FLOAT to INTEGER."}]
回答1件
0
ベストアンサー
csvファイルの中にテーブル定義と合わない無効なレコードがあると言われています。ファイルの185行目ですね。
NumberformatExceptionが出ているので、本来数値が入る箇所に文字列が入っている、という事はないでしょうか。
投稿2018/11/21 07:43
総合スコア89
あなたの回答
tips
プレビュー
バッドをするには、ログインかつ
こちらの条件を満たす必要があります。
2018/11/21 08:10
2018/11/21 08:13
2018/11/21 08:59
2018/11/21 09:10
2018/11/21 09:27
2018/11/21 10:09
2018/11/26 00:55
2018/11/26 05:04
2018/11/26 08:06
2018/11/26 08:34